Skip to content

Conversation

@nvazquez
Copy link
Contributor

@nvazquez nvazquez commented Sep 7, 2025

Description

This PR introduces improvements on the VMware to KVM migrations:

  • Add prefix log entries with the source VM name on the agent.log logs on the conversion hosts.

  • Improve the feedback to the end user by listing the in progress migration tasks (running and completed)

    • A new API listImportVmTasks is created. By default the running tasks are retrieved
    • When passing showcompleted = true to the listImportVmTasks then the completed tasks are retrieved
Screenshot 2025-09-19 at 12 43 35 AM
  • Allow passing extra parameters for virt-v2v conversion hosts if the administrator allows via global setting
    • A new global setting convert.vmware.instance.to.kvm.extra.params.allowed is created. Disabled by default.
    • When enabled, then its allowed to send extraparams on the importVm API, which are passed to the virt-v2v command executed on the conversion hosts.
Screenshot 2025-09-19 at 12 46 08 AM
  • Add the option to write directly to a storage pool on the virt-v2v conversion, instead of using a temporary location.
    • The default behavior is preserved, also when the parameter forceconverttopool = false is set on the importVm API. In this case the virt-v2v conversion will write to a temporary location and then CloudStack will move the disks to the conversion storage pool.
    • When the parameter forceconverttopool = true is set on the importVm API, then the selected conversion storage pool will be used to write directly on the virt-v2v conversion as the destination location. CloudStack will not move disks from a temporary location to the storage pool.
Screenshot 2025-09-19 at 12 47 25 AM

Types of changes

  • Breaking change (fix or feature that would cause existing functionality to change)
  • New feature (non-breaking change which adds functionality)
  • Bug fix (non-breaking change which fixes an issue)
  • Enhancement (improves an existing feature and functionality)
  • Cleanup (Code refactoring and cleanup, that may add test cases)
  • build/CI
  • test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

  • Major
  • Minor

Bug Severity

  • BLOCKER
  • Critical
  • Major
  • Minor
  • Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

@nvazquez nvazquez added this to the 4.22.0 milestone Sep 7, 2025
@codecov
Copy link

codecov bot commented Sep 7, 2025

Codecov Report

❌ Patch coverage is 19.51613% with 499 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.56%. Comparing base (b99a030) to head (dac73f4).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...ema/src/main/java/com/cloud/vm/ImportVMTaskVO.java 0.00% 124 Missing ⚠️
...apache/cloudstack/vm/ImportVmTasksManagerImpl.java 0.00% 123 Missing ⚠️
.../cloudstack/api/response/ImportVMTaskResponse.java 0.00% 109 Missing ⚠️
...ack/api/command/admin/vm/ListImportVMTasksCmd.java 0.00% 32 Missing ⚠️
.../apache/cloudstack/vm/UnmanagedVMsManagerImpl.java 73.33% 21 Missing and 11 partials ⚠️
...ain/java/com/cloud/vm/dao/ImportVMTaskDaoImpl.java 0.00% 23 Missing ⚠️
...ervisor/kvm/resource/LibvirtComputingResource.java 5.26% 18 Missing ⚠️
.../wrapper/LibvirtConvertInstanceCommandWrapper.java 60.52% 14 Missing and 1 partial ⚠️
...va/com/cloud/agent/api/ConvertInstanceCommand.java 18.18% 9 Missing ⚠️
...e/cloudstack/api/command/admin/vm/ImportVmCmd.java 0.00% 6 Missing ⚠️
... and 2 more
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #11594      +/-   ##
============================================
- Coverage     17.56%   17.56%   -0.01%     
- Complexity    15500    15518      +18     
============================================
  Files          5899     5906       +7     
  Lines        527793   528377     +584     
  Branches      64479    64528      +49     
============================================
+ Hits          92714    92785      +71     
- Misses       424653   425160     +507     
- Partials      10426    10432       +6     
Flag Coverage Δ
uitests 3.59% <ø> (-0.01%) ⬇️
unittests 18.62% <19.51%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@nvazquez
Copy link
Contributor Author

nvazquez commented Sep 7, 2025

@blueorangutan package

@blueorangutan
Copy link

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 14885

@nvazquez
Copy link
Contributor Author

@blueorangutan package

@nvazquez nvazquez marked this pull request as ready for review September 19, 2025 03:26
@blueorangutan
Copy link

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 15089

@nvazquez
Copy link
Contributor Author

@blueorangutan package

@blueorangutan
Copy link

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@nvazquez
Copy link
Contributor Author

@blueorangutan package

@blueorangutan
Copy link

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 15379

@nvazquez
Copy link
Contributor Author

@blueorangutan package

@blueorangutan
Copy link

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15380

@nvazquez
Copy link
Contributor Author

@blueorangutan test

@blueorangutan
Copy link

@nvazquez a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@github-actions
Copy link

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

@blueorangutan
Copy link

[SF] Trillian test result (tid-14600)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 112472 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11594-t14600-kvm-ol8.zip
Smoke tests completed. 109 look OK, 40 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
ContextSuite context=TestClusterDRS>:setup Error 0.00 test_cluster_drs.py
ContextSuite context=TestPurgeExpungedVms>:setup Error 0.00 test_purge_expunged_vms.py
test_another_user_can_allocate_ip_after_quarantined_has_ended_network Error 8.39 test_quarantined_ips.py
test_another_user_can_allocate_ip_after_quarantined_has_ended_vpc Error 0.42 test_quarantined_ips.py
test_only_owner_can_allocate_ip_in_quarantine_network Error 0.39 test_quarantined_ips.py
test_only_owner_can_allocate_ip_in_quarantine_vpc Error 0.44 test_quarantined_ips.py
test_CRUD_operations_userdata Error 1522.09 test_register_userdata.py
test_deploy_vm_with_registered_userdata Error 8.25 test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_allow Error 8.65 test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_append Error 8.79 test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_override_policy_deny Error 8.33 test_register_userdata.py
test_deploy_vm_with_registered_userdata_with_params Error 8.60 test_register_userdata.py
test_link_and_unlink_userdata_to_template Error 8.28 test_register_userdata.py
test_user_userdata_crud Error 8.26 test_register_userdata.py
ContextSuite context=TestResetVmOnReboot>:setup Error 0.00 test_reset_vm_on_reboot.py
test_02_create_template_with_checksum_sha1 Error 65.76 test_templates.py
test_03_create_template_with_checksum_sha256 Error 65.76 test_templates.py
test_04_create_template_with_checksum_md5 Error 65.66 test_templates.py
test_05_create_template_with_no_checksum Error 65.70 test_templates.py
test_02_deploy_vm_from_direct_download_template Error 1.51 test_templates.py
ContextSuite context=TestCreateTemplateWithDirectDownload>:teardown Error 11.96 test_templates.py
ContextSuite context=TestTemplates>:setup Error 25.72 test_templates.py
ContextSuite context=TestRAMCPUResourceAccounting>:setup Error 0.00 test_resource_accounting.py
ContextSuite context=TestResourceNames>:setup Error 0.00 test_resource_names.py
ContextSuite context=TestRestoreVM>:setup Error 0.00 test_restore_vm.py
ContextSuite context=TestRouterDHCPHosts>:setup Error 0.00 test_router_dhcphosts.py
ContextSuite context=TestRouterDHCPOpts>:setup Error 0.00 test_router_dhcphosts.py
ContextSuite context=TestRouterDns>:setup Error 0.00 test_router_dns.py
test_1_userdata_on_ssvm Failure 0.14 test_systemvm_userdata.py
test_2_userdata_on_cpvm Failure 0.12 test_systemvm_userdata.py
test_3_userdata_on_vr Error 4.74 test_systemvm_userdata.py
ContextSuite context=TestRouterDnsService>:setup Error 0.00 test_router_dnsservice.py
ContextSuite context=TestVMWareStoragePolicies>:setup Error 0.00 test_storage_policy.py
ContextSuite context=TestRouterIpTablesPolicies>:setup Error 0.00 test_routers_iptables_default_policy.py
ContextSuite context=TestVPCIpTablesPolicies>:setup Error 0.00 test_routers_iptables_default_policy.py
ContextSuite context=TestIsolatedNetworks>:setup Error 0.00 test_routers_network_ops.py
ContextSuite context=TestRedundantIsolateNetworks>:setup Error 0.00 test_routers_network_ops.py
ContextSuite context=TestRouterServices>:setup Error 0.00 test_routers.py
test_01_sys_vm_start Failure 0.13 test_secondary_storage.py
ContextSuite context=TestCpuCapServiceOfferings>:setup Error 0.00 test_service_offerings.py
ContextSuite context=TestServiceOfferings>:setup Error 0.41 test_service_offerings.py
ContextSuite context=TestSetSourceNatIp>:setup Error 0.00 test_set_sourcenat.py
ContextSuite context=TestSharedFSLifecycle>:setup Error 0.00 test_sharedfs_lifecycle.py
ContextSuite context=TestISOUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestLBRuleUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestNatRuleUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestPublicIPUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestSnapshotUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestVmUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestVolumeUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestVpnUsage>:setup Error 0.00 test_usage.py
ContextSuite context=TestSnapshotRootDisk>:setup Error 0.00 test_snapshots.py
ContextSuite context=TestSnapshotStandaloneBackup>:setup Error 0.00 test_snapshots.py
ContextSuite context=TestVmAutoScaling>:setup Error 0.00 test_vm_autoscaling.py
ContextSuite context=TestSslOffloading>:setup Error 0.00 test_ssl_offloading.py
test_01_list_sec_storage_vm Failure 0.06 test_ssvm.py
test_02_list_cpvm_vm Failure 0.05 test_ssvm.py
test_03_ssvm_internals Failure 0.06 test_ssvm.py
test_04_cpvm_internals Failure 0.05 test_ssvm.py
test_05_stop_ssvm Failure 0.06 test_ssvm.py
test_06_stop_cpvm Failure 0.05 test_ssvm.py
test_07_reboot_ssvm Failure 0.06 test_ssvm.py
test_08_reboot_cpvm Failure 0.06 test_ssvm.py
test_09_reboot_ssvm_forced Failure 0.05 test_ssvm.py
test_10_reboot_cpvm_forced Failure 0.05 test_ssvm.py
test_11_destroy_ssvm Failure 0.06 test_ssvm.py
test_12_destroy_cpvm Failure 0.05 test_ssvm.py
test_01_deploy_vm_on_specific_host Error 1.74 test_vm_deployment_planner.py
test_02_deploy_vm_on_specific_cluster Error 1.49 test_vm_deployment_planner.py
test_03_deploy_vm_on_specific_pod Error 1.59 test_vm_deployment_planner.py
test_04_deploy_vm_on_host_override_pod_and_cluster Error 1.42 test_vm_deployment_planner.py
test_05_deploy_vm_on_cluster_override_pod Error 1.45 test_vm_deployment_planner.py
ContextSuite context=TestDeployVM>:setup Error 0.00 test_vm_life_cycle.py
test_01_migrate_VM_and_root_volume Error 1.43 test_vm_life_cycle.py
test_02_migrate_VM_with_two_data_disks Error 1.39 test_vm_life_cycle.py
test_01_secure_vm_migration Error 81.14 test_vm_life_cycle.py
test_02_unsecure_vm_migration Error 223.65 test_vm_life_cycle.py
test_03_secured_to_nonsecured_vm_migration Error 151.71 test_vm_life_cycle.py
test_04_nonsecured_to_secured_vm_migration Error 152.84 test_vm_life_cycle.py
ContextSuite context=TestVMLifeCycle>:setup Error 8.08 test_vm_life_cycle.py
test_01_unmanage_vm_cycle_persistent_domain Error 4.10 test_vm_lifecycle_unmanage_kvm_import.py
test_02_unmanage_stopped_vm_cycle_persistent_domain Error 5.94 test_vm_lifecycle_unmanage_kvm_import.py
ContextSuite context=TestUnmanageKvmVM>:teardown Error 5.98 test_vm_lifecycle_unmanage_kvm_import.py
ContextSuite context=TestVMSchedule>:setup Error 0.00 test_vm_schedule.py
ContextSuite context=TestVmSnapshot>:setup Error 7.62 test_vm_snapshots.py
ContextSuite context=TestMigrateVMStrictTags>:setup Error 0.00 test_vm_strict_host_tags.py
ContextSuite context=TestRestoreVMStrictTags>:setup Error 0.00 test_vm_strict_host_tags.py
ContextSuite context=TestScaleVMStrictTags>:setup Error 0.00 test_vm_strict_host_tags.py
ContextSuite context=TestVMDeploymentPlannerStrictTags>:setup Error 0.00 test_vm_strict_host_tags.py
test_01_register_vnf_template Error 0.13 test_vnf_templates.py
test_02_list_vnf_template Error 0.14 test_vnf_templates.py
test_03_edit_vnf_template Error 0.10 test_vnf_templates.py
test_04_deploy_vnf_appliance Error 0.11 test_vnf_templates.py
test_05_delete_vnf_template Error 0.08 test_vnf_templates.py
ContextSuite context=TestCreateVolume>:setup Error 0.00 test_volumes.py
test_01_root_volume_encryption Error 1.85 test_volumes.py
test_02_data_volume_encryption Error 1.43 test_volumes.py
test_03_root_and_data_volume_encryption Error 1.48 test_volumes.py
ContextSuite context=TestVolumes>:setup Error 29.15 test_volumes.py
ContextSuite context=TestIpv6Vpc>:setup Error 0.00 test_vpc_ipv6.py
ContextSuite context=TestVPCRedundancy>:setup Error 0.00 test_vpc_redundant.py
ContextSuite context=TestVPCNics>:setup Error 0.00 test_vpc_router_nics.py
ContextSuite context=TestRVPCSite2SiteVpn>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVPCSite2SiteVPNMultipleOptions>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVpcRemoteAccessVpn>:setup Error 0.00 test_vpc_vpn.py
ContextSuite context=TestVpcSite2SiteVpn>:setup Error 0.00 test_vpc_vpn.py
test_02_cancel_host_maintenace_with_migration_jobs Error 3.19 test_host_maintenance.py
test_03_cancel_host_maintenace_with_migration_jobs_failure Error 4.97 test_host_maintenance.py
test_01_cancel_host_maintenance_ssh_enabled_agent_connected Failure 21.98 test_host_maintenance.py
test_03_cancel_host_maintenance_ssh_disabled_agent_connected Failure 19.15 test_host_maintenance.py
test_04_cancel_host_maintenance_ssh_disabled_agent_disconnected Failure 33.82 test_host_maintenance.py
ContextSuite context=TestHostMaintenanceAgents>:teardown Error 35.03 test_host_maintenance.py
test_disable_oobm_ha_state_ineligible Error 1528.21 test_hostha_kvm.py

@blueorangutan
Copy link

[SF] Trillian test result (tid-14604)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 58688 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11594-t14604-kvm-ol8.zip
Smoke tests completed. 140 look OK, 9 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File
ContextSuite context=TestRouterDns>:setup Error 0.00 test_router_dns.py
ContextSuite context=TestRouterDnsService>:setup Error 0.00 test_router_dnsservice.py
ContextSuite context=TestRouterIpTablesPolicies>:setup Error 0.00 test_routers_iptables_default_policy.py
ContextSuite context=TestVPCIpTablesPolicies>:setup Error 0.00 test_routers_iptables_default_policy.py
test_01_isolate_network_FW_PF_default_routes_egress_true Error 0.28 test_routers_network_ops.py
test_02_isolate_network_FW_PF_default_routes_egress_false Error 0.29 test_routers_network_ops.py
ContextSuite context=TestRedundantIsolateNetworks>:setup Error 1.60 test_routers_network_ops.py
ContextSuite context=TestRouterServices>:setup Error 0.00 test_routers.py
ContextSuite context=TestCpuCapServiceOfferings>:setup Error 0.00 test_service_offerings.py
ContextSuite context=TestServiceOfferings>:setup Error 0.39 test_service_offerings.py
ContextSuite context=TestSetSourceNatIp>:setup Error 0.00 test_set_sourcenat.py
ContextSuite context=TestSnapshotRootDisk>:setup Error 0.00 test_snapshots.py
ContextSuite context=TestSnapshotStandaloneBackup>:setup Error 0.00 test_snapshots.py
ContextSuite context=TestSslOffloading>:setup Error 0.00 test_ssl_offloading.py

@nvazquez
Copy link
Contributor Author

@blueorangutan package

@blueorangutan
Copy link

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Copy link
Contributor

@NuxRo NuxRo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, tested with an actual VMWare to KVM migration using extra parameters "--mac 00:0c:29:e6:3d:9d:ip:192.168.0.89,192.168.0.1,24,192.168.0.254 -x"

@blueorangutan
Copy link

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15412

@nvazquez
Copy link
Contributor Author

@blueorangutan test

@blueorangutan
Copy link

@nvazquez a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@nvazquez nvazquez merged commit b106d6e into apache:main Oct 10, 2025
27 of 28 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in Apache CloudStack 4.22.0 Oct 10, 2025
@blueorangutan
Copy link

[SF] Trillian test result (tid-14632)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 55358 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11594-t14632-kvm-ol8.zip
Smoke tests completed. 149 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test Result Time (s) Test File

@DaanHoogland DaanHoogland deleted the 422-vmware-to-kvm-improvements branch October 13, 2025 06:31
dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Oct 17, 2025
* Add source VM name on virt-v2v migration log entries

* Improve the feedback by displaying the running importing tasks

* Add source VM name prefix on more conversion logs

* Improve listing and also list completed tasks

* Pass extra parameters to virt-v2v if administrator allows via global setting

* Add Force converting directly to storage pool option

* Refactor based on review comments

* Add properties for env vars for the instance conversion

* Add separate component for Import VM Tasks

* applying copilot suggestions from code review

Co-authored-by: Copilot <[email protected]>

* Fix importing unmanaged instances due to incorrect internal name

* Add VM prefix on each log operation for conversion

* Log the original VM name instead of the cloned VM in case of cloning

* Allow searching storage pool by UUID after conversion to support SharedMountPoint

* Fix search pools logic

* Improve UI and add checks for force convert to pool parameter

* Support Local storage when forceconverttopool is set to true

* Add config key to for allowed extra params and add validation

* Fix params lists

* Fix compile error

* Remove extra stubbings

* Fix extra params execution

---------

Co-authored-by: Abhishek Kumar <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Suresh Kumar Anaparti <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

9 participants