Releases: zilliztech/milvus-backup
v0.5.9
What's Changed
- core: return error msg when fail by @huanghaoyuanhhy in #753
- storage, paramtable : support alicloud by @huanghaoyuanhhy in #755
- meta: support read from level file by @huanghaoyuanhhy in #752
- *: move server to core by @huanghaoyuanhhy in #756
- build(deps): bump github.com/go-viper/mapstructure/v2 from 2.3.0 to 2.4.0 by @dependabot[bot] in #757
- build(deps): bump actions/checkout from 4 to 5 by @dependabot[bot] in #760
- storage: improve azure paging logic by @huanghaoyuanhhy in #759
- *: refactor for support filter option by @huanghaoyuanhhy in #761
- storage: use original context for semaphore acquisition by @huanghaoyuanhhy in #762
- storage: retry when azure copy object fail by @huanghaoyuanhhy in #763
- cmd/restore: rename restore_index to rebuild_index by @huanghaoyuanhhy in #764
- cmd/server: fix validate and add ut by @huanghaoyuanhhy in #766
- *: support get meta size by @huanghaoyuanhhy in #765
- readme: add milvus 2.6 by @huanghaoyuanhhy in #768
- *: make golangci-lint happy by @huanghaoyuanhhy in #769
- ci: add azure storage test pipeline by @zhuwenxing in #772
- test: use sentence-transformers/all-MiniLM-L6-v2 model for ci by @zhuwenxing in #773
- build(deps): bump deepdiff from 6.7.1 to 8.6.1 in /tests by @dependabot[bot] in #774
- storage: return file info directly when prefix is a file by @huanghaoyuanhhy in #780
- cmd/restore: add --filter option by @huanghaoyuanhhy in #779
- build(deps): bump actions/setup-go from 5 to 6 by @dependabot[bot] in #776
- build(deps): bump actions/setup-python from 5 to 6 by @dependabot[bot] in #777
- storage, proto: format code by @huanghaoyuanhhy in #785
- fix(log): correct file logging config by @rere950303 in #783
- configs, log: use
filenameinstead ofrootPathby @huanghaoyuanhhy in #788 - *: support read backup root path from request by @huanghaoyuanhhy in #787
- docs: Point to env_variables.md in README.md by @neelaundhia in #794
- restore, storage: support keep temp file by @huanghaoyuanhhy in #795
- ci: remove pull_request trigger from azure workflow to avoid secrets access issues by @zhuwenxing in #796
- meta, backup: add task id to meta map by @huanghaoyuanhhy in #798
- proto: support new datatype by @huanghaoyuanhhy in #802
- backup: use collection semaphore instead of copy semaphore by @huanghaoyuanhhy in #803
- go.mod: update dep by @huanghaoyuanhhy in #805
- server: register pprof router by @huanghaoyuanhhy in #804
- backup: improve log format by @huanghaoyuanhhy in #806
- backup, restore: support struct array by @huanghaoyuanhhy in #808
- proto: remove unused message by @huanghaoyuanhhy in #811
- *: move taskmgr to internal by @huanghaoyuanhhy in #810
- test: add geometry datatype in test by @zhuwenxing in #809
- *: move storage to internal by @huanghaoyuanhhy in #813
- build(deps): bump actions/upload-artifact from 4 to 5 by @dependabot[bot] in #815
- release v0.5.9 by @huanghaoyuanhhy in #820
New Contributors
- @rere950303 made their first contribution in #783
- @neelaundhia made their first contribution in #794
Full Changelog: v0.5.8...v0.5.9
v0.5.9-rc1
What's Changed
- core: return error msg when fail by @huanghaoyuanhhy in #753
- storage, paramtable : support alicloud by @huanghaoyuanhhy in #755
- meta: support read from level file by @huanghaoyuanhhy in #752
- *: move server to core by @huanghaoyuanhhy in #756
- build(deps): bump github.com/go-viper/mapstructure/v2 from 2.3.0 to 2.4.0 by @dependabot[bot] in #757
- build(deps): bump actions/checkout from 4 to 5 by @dependabot[bot] in #760
- storage: improve azure paging logic by @huanghaoyuanhhy in #759
- *: refactor for support filter option by @huanghaoyuanhhy in #761
- storage: use original context for semaphore acquisition by @huanghaoyuanhhy in #762
- storage: retry when azure copy object fail by @huanghaoyuanhhy in #763
- cmd/restore: rename restore_index to rebuild_index by @huanghaoyuanhhy in #764
- cmd/server: fix validate and add ut by @huanghaoyuanhhy in #766
- *: support get meta size by @huanghaoyuanhhy in #765
- readme: add milvus 2.6 by @huanghaoyuanhhy in #768
- *: make golangci-lint happy by @huanghaoyuanhhy in #769
- ci: add azure storage test pipeline by @zhuwenxing in #772
- test: use sentence-transformers/all-MiniLM-L6-v2 model for ci by @zhuwenxing in #773
- build(deps): bump deepdiff from 6.7.1 to 8.6.1 in /tests by @dependabot[bot] in #774
- storage: return file info directly when prefix is a file by @huanghaoyuanhhy in #780
- cmd/restore: add --filter option by @huanghaoyuanhhy in #779
- build(deps): bump actions/setup-go from 5 to 6 by @dependabot[bot] in #776
- build(deps): bump actions/setup-python from 5 to 6 by @dependabot[bot] in #777
- storage, proto: format code by @huanghaoyuanhhy in #785
- fix(log): correct file logging config by @rere950303 in #783
- configs, log: use
filenameinstead ofrootPathby @huanghaoyuanhhy in #788 - *: support read backup root path from request by @huanghaoyuanhhy in #787
- docs: Point to env_variables.md in README.md by @neelaundhia in #794
- restore, storage: support keep temp file by @huanghaoyuanhhy in #795
- ci: remove pull_request trigger from azure workflow to avoid secrets access issues by @zhuwenxing in #796
- meta, backup: add task id to meta map by @huanghaoyuanhhy in #798
- proto: support new datatype by @huanghaoyuanhhy in #802
- backup: use collection semaphore instead of copy semaphore by @huanghaoyuanhhy in #803
- go.mod: update dep by @huanghaoyuanhhy in #805
- server: register pprof router by @huanghaoyuanhhy in #804
- backup: improve log format by @huanghaoyuanhhy in #806
- backup, restore: support struct array by @huanghaoyuanhhy in #808
- proto: remove unused message by @huanghaoyuanhhy in #811
- *: move taskmgr to internal by @huanghaoyuanhhy in #810
- test: add geometry datatype in test by @zhuwenxing in #809
- *: move storage to internal by @huanghaoyuanhhy in #813
- build(deps): bump actions/upload-artifact from 4 to 5 by @dependabot[bot] in #815
New Contributors
- @rere950303 made their first contribution in #783
- @neelaundhia made their first contribution in #794
Full Changelog: v0.5.8...v0.5.9-rc1
v0.5.8
What's Changed
- *: abstract filter and mapping logic into planner for task generation by @huanghaoyuanhhy in #726
- paramtable, storage: support alibaba by @huanghaoyuanhhy in #731
- restore: parallel restore L0 segment by @huanghaoyuanhhy in #711
- restore, paramtable: parallelize BulkImport tasks by @huanghaoyuanhhy in #733
- backup: parallel backup segments by @huanghaoyuanhhy in #729
- *: support int8 vector by @huanghaoyuanhhy in #734
- *: remove unused code by @huanghaoyuanhhy in #736
- readme: add compatibility and v2 restore mode by @huanghaoyuanhhy in #737
- readme: recommend using latest milvus-backup release in README by @huanghaoyuanhhy in #738
- cloud, migrate: use new migrate api by @huanghaoyuanhhy in #739
- *: fix log params by @huanghaoyuanhhy in #740
- *: support backup and restore v2 segment by @huanghaoyuanhhy in #741
- test: enable storage v2 for backup&restore by @zhuwenxing in #732
- Create env_variables.md by @jaelgu in #743
- .goreleaser: add version by @huanghaoyuanhhy in #749
New Contributors
Full Changelog: v0.5.7...v0.5.8
v0.5.8-rc1
What's Changed
- *: abstract filter and mapping logic into planner for task generation by @huanghaoyuanhhy in #726
- paramtable, storage: support alibaba by @huanghaoyuanhhy in #731
- restore: parallel restore L0 segment by @huanghaoyuanhhy in #711
- restore, paramtable: parallelize BulkImport tasks by @huanghaoyuanhhy in #733
- backup: parallel backup segments by @huanghaoyuanhhy in #729
- *: support int8 vector by @huanghaoyuanhhy in #734
- *: remove unused code by @huanghaoyuanhhy in #736
- readme: add compatibility and v2 restore mode by @huanghaoyuanhhy in #737
- readme: recommend using latest milvus-backup release in README by @huanghaoyuanhhy in #738
- cloud, migrate: use new migrate api by @huanghaoyuanhhy in #739
- *: fix log params by @huanghaoyuanhhy in #740
- *: support backup and restore v2 segment by @huanghaoyuanhhy in #741
- test: enable storage v2 for backup&restore by @zhuwenxing in #732
- Create env_variables.md by @jaelgu in #743
New Contributors
Full Changelog: v0.5.7...v0.5.8-rc1
v0.5.7
What's Changed
- milvus: retry on Bulkinsert task status fetch failure by @huanghaoyuanhhy in #716
- storage, migrate: support migrate to zilliz via ali by @huanghaoyuanhhy in #723
- cloud: hiding confidential information in logs by @huanghaoyuanhhy in #724
- storage: use azureAccountName directly in URL instead of accessKeyID by @huanghaoyuanhhy in #727
- *: support auto-renewal of GC pause during backup by @huanghaoyuanhhy in #718
- milvus: skip retry if server not support get segment info by @huanghaoyuanhhy in #730
Full Changelog: v0.5.7-rc1...v0.5.7
v0.5.7-rc1
What's Changed
- restore: edit error msg by @huanghaoyuanhhy in #676
- *: refine storage pkg by @huanghaoyuanhhy in #673
- *: remove some unnecessary dependencies by @huanghaoyuanhhy in #683
- storage: use GetCredential to get ali cred by @huanghaoyuanhhy in #684
- backup: skip describe db if not support by @huanghaoyuanhhy in #687
- ci: add 2.2 and 2.3 as source tag for cross version br by @zhuwenxing in #681
- backup, client: check feature before backup by @huanghaoyuanhhy in #688
- build(deps): bump requests from 2.32.0 to 2.32.4 in /tests by @dependabot in #689
- Parallelize restore v1 bulk inserts by @sonakshireddym in #675
- Get backup field values in backup.yaml from environment variables by @sonakshireddym in #693
- *: support migrate to zilliz by @huanghaoyuanhhy in #690
- storage: improve copy retry logic by @huanghaoyuanhhy in #694
- backup: improve backup rbac logic by @huanghaoyuanhhy in #696
- migrate, storage: re apply stage cred when expire by @huanghaoyuanhhy in #698
- milvus: retry GetPersistentSegmentInfo when return error by @huanghaoyuanhhy in #699
- params, storage: add region and token config by @huanghaoyuanhhy in #700
- *: estimate copy to cloud speed and remaining time by @huanghaoyuanhhy in #704
- restore: fix typo by @huanghaoyuanhhy in #706
- test: add property check after restore by @zhuwenxing in #707
- backup, restore: support br collection properties by @huanghaoyuanhhy in #708
- storage: pass the region parameter to the newClient function by @huanghaoyuanhhy in #709
- test: add more case for useV2Restore param by @zhuwenxing in #712
- build(deps): bump github.com/go-viper/mapstructure/v2 from 2.2.1 to 2.3.0 by @dependabot in #713
- storage: fix azure list iter by @huanghaoyuanhhy in #715
New Contributors
- @sonakshireddym made their first contribution in #675
Full Changelog: v0.5.6...v0.5.7-rc1
v0.5.6
What's Changed
- *: update dep by @huanghaoyuanhhy in #650
- storage: remove list object operation from copy function by @huanghaoyuanhhy in #651
- cmd: return 1 when fail by @huanghaoyuanhhy in #653
- storage: skip empty file when copy by @huanghaoyuanhhy in #654
- *: fix restore progress tracking by @huanghaoyuanhhy in #657
- cmd: export v2 restore flag by @huanghaoyuanhhy in #658
- taskmgr: add read lock by @huanghaoyuanhhy in #660
- restore, client: improve retry logic by @huanghaoyuanhhy in #665
- backup, client: improve flush logic by @huanghaoyuanhhy in #666
- backup: improve rpc backup error msg by @huanghaoyuanhhy in #667
- backup: read gc addr from backup cfg by @huanghaoyuanhhy in #669
- backup: fix check flush by @huanghaoyuanhhy in #671
- client: use outside ctx by @huanghaoyuanhhy in #672
Full Changelog: v0.5.5...v0.5.6
v0.5.5
⚠️ Breaking Change (Carried over from previous version)
Previously, the milvus-backup tool only backed up databases that contained at least one collection.
Now, it backs up all databases, including those without any collections, and also includes database-level parameters (such as database.diskQuota.mb and database.replica.number ).
⚠️ Requires additional permission:DescribeDatabase— the backup will fail without it.
Additionally:
- Backup files now include metadata for empty databases and their parameters;
- Restoring these parameters is still not supported — only backup is performed;
- During restore, the empty databases will be recreated.
What's Changed
- ci: use custom config for deployment by @zhuwenxing in #631
- backup, restore: use base64 to encode default value proto by @huanghaoyuanhhy in #646
- fix: update golang.org/x/oauth2 to address CVE-2025-22868 by @songohannyc in #645
- backup: sort by field id by @huanghaoyuanhhy in #648
- client: add identifier to outgoing ctx by @huanghaoyuanhhy in #644
- restore: fix skip rbac not work by @huanghaoyuanhhy in #649
New Contributors
- @songohannyc made their first contribution in #645
Full Changelog: v0.5.4...v0.5.5
v0.5.4
⚠️ Breaking Change
Starting from this release, the milvus-backup tool will backup all databases, even if a database contains no collections, and will also backup parameters associated with each database.
Implications:
-
Requires additional permission: DescribeDatabase — missing this permission may cause the backup process to fail.
-
The backup file will include metadata for empty databases and their parameters;
-
During restore, only the databases will be recreated — parameters will not be restored yet;
Please ensure proper permissions are granted and review any scripts or workflows that may rely on the previous behavior before upgrading.
What's Changed
- fix: Fix the bug that unable to restore json path index by @sunby in #608
- backup, restore: support backup and restore privilege group by @huanghaoyuanhhy in #612
- test: add rbac restore testcases by @zhuwenxing in #615
- ci: fix get latest tag by @zhuwenxing in #619
- contributing.md: add by @huanghaoyuanhhy in #618
- test: add dynamic data for insert by @zhuwenxing in #620
- backup: print logs when reading rbac info done by @huanghaoyuanhhy in #621
- build(deps): bump golang.org/x/net from 0.36.0 to 0.38.0 by @dependabot in #622
- test: enable backup test cases for json path index after code fix by @binbinlv in #610
- *: read binlog file list from proxy node by @huanghaoyuanhhy in #609
- restore: compatible db backup is empty by @huanghaoyuanhhy in #624
- milvus-backup failure for Azure Blob Storage Account by @gifi-siby in #496
- restore: build task before run by @huanghaoyuanhhy in #626
- gcpnative support for backup tool (GCS implemented using Google Cloud Storage libraries) by @gifi-siby in #625
- restore: filter backup after rename by @huanghaoyuanhhy in #627
- backup: fix backup binlog not append by @huanghaoyuanhhy in #630
- mpath: allow empty root dir by @huanghaoyuanhhy in #632
- *: add some ut by @huanghaoyuanhhy in #634
- backup: verify binlog number when run backup by @huanghaoyuanhhy in #636
New Contributors
Full Changelog: v0.5.3...v0.5.4
v0.5.3
What's Changed
- restore: fix bug where chunked segment was processed but not used by @huanghaoyuanhhy in #559
- ci: refine get_latest_tag.py script by @zhuwenxing in #565
- restore: truncate binlog by timestamp only when specified by @huanghaoyuanhhy in #567
- build(deps): bump golang.org/x/net from 0.33.0 to 0.36.0 by @dependabot in #568
- client: add aimd limit for create by @huanghaoyuanhhy in #569
- ci: update dependabot.yml to include Go and Python dependencies and fix some cve by @zhuwenxing in #572
- restore: restore only meta if meta-only flag is set by @huanghaoyuanhhy in #573
- core, client: improve log readability during the flush process by @huanghaoyuanhhy in #575
- client: flush wait condition no longer checks empty seg IDs by @huanghaoyuanhhy in #579
- test: add bm25 function in delete upsert case by @zhuwenxing in #574
- test: add text embedding function test by @zhuwenxing in #576
- restore: fix dropCollection flag not work by @huanghaoyuanhhy in #580
- ci: update mergify config and remove dependabot config by @zhuwenxing in #602
- build(deps): bump github.com/golang-jwt/jwt/v5 from 5.2.1 to 5.2.2 by @dependabot in #603
- test: Add test cases for json path index backup by @binbinlv in #604
- ci: fix nightly for text embedding function test by @zhuwenxing in #607
New Contributors
Full Changelog: v0.5.2...v0.5.3