Fix removeUsage for backups #11522

abh1sar · 2025-08-27T05:11:15Z

Description

Due to a bug in UsageBackupDaoImpl.removeUsage, entry in the cloud_usage.backup_usage doesn't get removed even after all the backups have been deleted and backup offering removed.
Backup usage doesn't get updated to 0 size when all backups are deleted.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Before Fix:

10:03:37.297 [Usage-Job-1] ERROR com.cloud.usage.dao.UsageBackupDaoImpl_EnhancerByCloudStack_c6b7eb21 - Exception caught while removing UsageBackupVO: Remove backup usage exception: No value specified for parameter 4
com.cloud.exception.CloudException: Remove backup usage exception: No value specified for parameter 4                                                                                     
        at com.cloud.usage.dao.UsageBackupDaoImpl.removeUsage(UsageBackupDaoImpl.java:76) ~[cloud-engine-schema-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]                                  
        at com.cloud.usage.dao.UsageBackupDaoImpl_EnhancerByCloudStack_c6b7eb21.CGLIB$removeUsage$0(<generated>) ~[cloud-engine-schema-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]           
        at com.cloud.usage.dao.UsageBackupDaoImpl_EnhancerByCloudStack_c6b7eb21_FastClassByCloudStack_515fb514.invoke(<generated>) ~[cloud-engine-schema-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]
        at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) ~[cglib-nodep-3.3.0.jar:?]                                                                                    
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:123) ~[cloud-utils-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]
        at com.cloud.usage.dao.UsageBackupDaoImpl_EnhancerByCloudStack_c6b7eb21.removeUsage(<generated>) ~[cloud-engine-schema-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]                   
        at com.cloud.usage.UsageManagerImpl.createBackupEvent(UsageManagerImpl.java:2077) ~[classes/:?]                                                                                   
        at com.cloud.usage.UsageManagerImpl.createHelperRecord(UsageManagerImpl.java:986) ~[classes/:?]                                                                                   
        at com.cloud.usage.UsageManagerImpl.parse(UsageManagerImpl.java:716) ~[classes/:?]                                                                                                
        at com.cloud.usage.UsageManagerImpl.runInContextInternal(UsageManagerImpl.java:432) ~[classes/:?]                                                                      
        at com.cloud.usage.UsageManagerImpl$1.runInContext(UsageManagerImpl.java:369) ~[classes/:?]                                                                                       
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) ~[cloud-framework-managed-context-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]                                                                                                                                                                                           
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) ~[cloud-framework-managed-context-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) ~[cloud-framework-managed-context-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) ~[cloud-framework-managed-context-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) ~[cloud-framework-managed-context-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]
        at com.cloud.usage.UsageManagerImpl.run(UsageManagerImpl.java:371) ~[classes/:?]                                                                                                  
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]                                                                                                
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]                                                                                                               
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]                                                                                                                                                                                                                                                   
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]                                                                                                                                                                                                                                                                                  
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]                                                                                                                                                                                                                                                                                  
        at java.lang.Thread.run(Thread.java:829) ~[?:?]                                                                                                                                  
Caused by: java.sql.SQLException: No value specified for parameter 4                                                                                                                     
        at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:130) ~[mysql-connector-j-8.0.33.jar:8.0.33]                                                                                                                                                                                                                                                      
        at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) ~[mysql-connector-j-8.0.33.jar:8.0.33]                                                                                                                                                                                                                              
        at com.mysql.cj.jdbc.ServerPreparedStatement.serverExecute(ServerPreparedStatement.java:555) ~[mysql-connector-j-8.0.33.jar:8.0.33]                                                                                                                                                                                                                                        
        at com.mysql.cj.jdbc.ServerPreparedStatement.executeInternal(ServerPreparedStatement.java:339) ~[mysql-connector-j-8.0.33.jar:8.0.33]                                                                                                                                                                                                                                      
        at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1061) ~[mysql-connector-j-8.0.33.jar:8.0.33]                                                                                                                                                                                                                               
        at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1009) ~[mysql-connector-j-8.0.33.jar:8.0.33]                                                                                                                                                                                                                               
        at com.mysql.cj.jdbc.ClientPreparedStatement.executeLargeUpdate(ClientPreparedStatement.java:1320) ~[mysql-connector-j-8.0.33.jar:8.0.33]                                                                                                                                                                                                                                  
        at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdate(ClientPreparedStatement.java:994) ~[mysql-connector-j-8.0.33.jar:8.0.33]                                                                                                                                                                                                                                        
        at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61) ~[HikariCP-5.1.0.jar:?]                                                                                                                                                                                                                                                     
        at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java) ~[HikariCP-5.1.0.jar:?]                                                                                                                                                                                                                                            
        at com.cloud.usage.dao.UsageBackupDaoImpl.removeUsage(UsageBackupDaoImpl.java:72) ~[cloud-engine-schema-4.21.0.0-SNAPSHOT.jar:4.21.0.0-SNAPSHOT]

After Fix:

Created two backups for a vm (id=8) with a backup_offering (id=1)
(Ignore usage_backup id 20 as it from a previous testing)

2. Deleted one backup

3. Deleted another backup

4. Removed backup offering

How did you try to break this feature and the system with this change?

abh1sar · 2025-08-27T05:14:43Z

@blueorangutan package

blueorangutan · 2025-08-27T05:16:03Z

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

codecov · 2025-08-27T05:19:25Z

Codecov Report

❌ Patch coverage is 40.00000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 17.42%. Comparing base (ec533cd) to head (0a9b232).
⚠️ Report is 14 commits behind head on main.

Files with missing lines	Patch %	Lines
...rg/apache/cloudstack/backup/BackupManagerImpl.java	50.00%	1 Missing and 1 partial ⚠️
...n/java/com/cloud/usage/dao/UsageBackupDaoImpl.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##               main   #11522   +/-   ##
=========================================
  Coverage     17.42%   17.42%           
- Complexity    15336    15339    +3     
=========================================
  Files          5892     5892           
  Lines        526521   526524    +3     
  Branches      64293    64294    +1     
=========================================
+ Hits          91767    91771    +4     
+ Misses       424401   424399    -2     
- Partials      10353    10354    +1

Flag	Coverage Δ
uitests	`3.61% <ø> (ø)`
unittests	`18.48% <40.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

weizhouapache

code lgtm

blueorangutan · 2025-08-27T06:26:38Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14738

winterhazel

CLGTM. Did not test

harikrishna-patnala

code LGTM

sureshanaparti

clgtm

abh1sar · 2025-09-26T11:58:58Z

@blueorangutan package

blueorangutan · 2025-09-26T12:00:04Z

@abh1sar a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-09-26T13:47:27Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 15187

borisstoyanov · 2025-10-01T09:41:59Z

@blueorangutan test keepEnv

blueorangutan · 2025-10-01T09:44:04Z

@borisstoyanov a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-10-02T02:25:26Z

[SF] Trillian test result (tid-14494)
Environment: kvm-ol8 (x2), zone: Advanced Networking with Mgmt server ol8
Total time taken: 56786 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11522-t14494-kvm-ol8.zip
Smoke tests completed. 147 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

borisstoyanov

LGTM. manually checked the fix and it seems to be working as expected

fix removeUsage for backups

eef76c2

boring-cyborg bot added the component:backup label Aug 27, 2025

weizhouapache approved these changes Aug 27, 2025

View reviewed changes

winterhazel approved these changes Aug 27, 2025

View reviewed changes

rohityadavcloud added this to the 4.22.0 milestone Aug 27, 2025

harikrishna-patnala approved these changes Aug 28, 2025

View reviewed changes

shwstppr mentioned this pull request Aug 28, 2025

4.21.0.0 release notes updates apache/cloudstack-documentation#549

Merged

sureshanaparti approved these changes Aug 28, 2025

View reviewed changes

sureshanaparti added the status:needs-testing label Aug 28, 2025

sureshanaparti added this to Apache CloudStack 4.22.0 Aug 28, 2025

sureshanaparti moved this to In Progress in Apache CloudStack 4.22.0 Aug 28, 2025

rosi-shapeblue self-assigned this Sep 2, 2025

abh1sar added this to Apache CloudStack BugFest - Issues Sep 26, 2025

abh1sar removed this from Apache CloudStack BugFest - Issues Sep 26, 2025

Merge branch 'main' into backup-usage

0a9b232

abh1sar added the Severity:Critical Critical bug label Sep 26, 2025

abh1sar unassigned rosi-shapeblue Oct 1, 2025

borisstoyanov approved these changes Oct 2, 2025

View reviewed changes

harikrishna-patnala merged commit 1efa46c into apache:main Oct 2, 2025
28 checks passed

harikrishna-patnala deleted the backup-usage branch October 2, 2025 11:45

github-project-automation bot moved this from In Progress to Done in Apache CloudStack 4.22.0 Oct 2, 2025

DaanHoogland removed the status:needs-testing label Oct 3, 2025

Fix removeUsage for backups #11522

Fix removeUsage for backups #11522

Uh oh!

Conversation

abh1sar commented Aug 27, 2025

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

abh1sar commented Aug 27, 2025

Uh oh!

blueorangutan commented Aug 27, 2025

Uh oh!

codecov bot commented Aug 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Aug 27, 2025

Uh oh!

winterhazel left a comment

Choose a reason for hiding this comment

Uh oh!

harikrishna-patnala left a comment

Choose a reason for hiding this comment

Uh oh!

sureshanaparti left a comment

Choose a reason for hiding this comment

Uh oh!

abh1sar commented Sep 26, 2025

Uh oh!

blueorangutan commented Sep 26, 2025

Uh oh!

blueorangutan commented Sep 26, 2025

Uh oh!

borisstoyanov commented Oct 1, 2025

Uh oh!

blueorangutan commented Oct 1, 2025

Uh oh!

blueorangutan commented Oct 2, 2025

Uh oh!

borisstoyanov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Aug 27, 2025 •

edited

Loading