-
Notifications
You must be signed in to change notification settings - Fork 201
docs: data injections migration guide #4068
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
AustinAbro321
wants to merge
7
commits into
main
Choose a base branch
from
data-injections-migration-guide
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
46caf62
data injections migration
AustinAbro321 c7869ba
oci image
AustinAbro321 f4615d5
host dependency
AustinAbro321 b24d570
folder
AustinAbro321 66d49a1
data injections
AustinAbro321 46cfd8d
alternative
AustinAbro321 8c2959e
my-app
AustinAbro321 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
117 changes: 117 additions & 0 deletions
117
site/src/content/docs/best-practices/data-injections-migration.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,117 @@ | ||
--- | ||
title: "Migrate off of Data injections" | ||
--- | ||
|
||
This guide explains how to migrate your Zarf deployment from the data injections feature to OCI image-based data delivery methods. Data injections is planned to be deprecated the current Zarf schema version and fully removed by Zarf v1.0.0. There are for several reasons behind this: | ||
|
||
- **Poor User Experience**: Many users have struggled to figure out how to adopt data injections. | ||
- **Host dependency**: Data injections shells out to `tar`. This makes testing difficult and introduces differences across environments. | ||
- **Better alternatives**: OCI images provide a Kubernetes native solution for data delivery, and neatly fit into the Zarf delivery paradigm. | ||
|
||
## Migration guide | ||
|
||
### Step 1: Package Your Data in an OCI Image | ||
|
||
First, create a container image containing your data: | ||
|
||
```dockerfile | ||
FROM alpine:3.18 | ||
COPY your-data-file /your-data/your-data-file | ||
``` | ||
|
||
Build and push this image: | ||
```bash | ||
docker build -t your-registry/your-data:tag . | ||
docker push your-registry/your-data:tag | ||
``` | ||
|
||
Some registries will not accept images over a certain size. In this case, you can load the image from the Docker Daemon. This can be slow, users are recommended to try out [this strategy for improving the speed of image loads](https://docs.zarf.dev/faq#how-can-i-improve-the-speed-of-loading-large-images-from-docker-on-zarf-package-create). Additionally, before Data Injections is removed Zarf will allow users to load images directly from a tar file, ([#2181](https://github.com/zarf-dev/zarf/issues/)), this will be significantly faster than Docker daemon pulls. | ||
|
||
### Step 2: Update zarf.yaml | ||
|
||
**Before (Data Injections):** | ||
```yaml | ||
kind: ZarfPackageConfig | ||
metadata: | ||
name: my-data-app | ||
|
||
components: | ||
- name: my-app | ||
required: true | ||
images: | ||
- ghcr.io/my-app:1.0.0 | ||
- alpine:3.18 | ||
dataInjections: | ||
- source: my-folder | ||
target: | ||
namespace: my-app | ||
selector: app=my-app | ||
container: data-loader | ||
path: /data | ||
compress: true | ||
``` | ||
|
||
**After (Init Container Strategy):** | ||
```yaml | ||
kind: ZarfPackageConfig | ||
metadata: | ||
name: init-data-loading | ||
|
||
components: | ||
- name: my-app | ||
required: true | ||
images: | ||
- ghcr.io/my-app:1.0.0 | ||
- your-registry/your-data:tag # Your container with your data file | ||
``` | ||
|
||
**Key Changes:** | ||
- Remove `dataInjections` section entirely. | ||
- Replace the image used during data injections with your data image. | ||
|
||
### Step 3: Update Deployment Manifest | ||
|
||
**Before (Data Injections):** | ||
```yaml | ||
spec: | ||
template: | ||
spec: | ||
initContainers: | ||
- name: data-loader | ||
image: alpine:3.18 | ||
command: ["sh", "-c"] | ||
args: | ||
- 'while [ ! -f /data/###ZARF_DATA_INJECTION_MARKER### ]; do echo "waiting for zarf data sync" && sleep 1; done; echo "we are done waiting!"' | ||
volumeMounts: | ||
- mountPath: /data | ||
name: data | ||
``` | ||
|
||
**After (Init Container Strategy):** | ||
```yaml | ||
spec: | ||
template: | ||
spec: | ||
initContainers: | ||
- name: data-puller | ||
image: your-registry/your-data:tag | ||
command: ["sh", "-c"] | ||
args: | ||
- | | ||
cp /your-data/your-data-file /data/my-app-data-location | ||
ls -la /data | ||
echo "Data initialization complete" | ||
volumeMounts: | ||
- mountPath: /data | ||
name: data | ||
``` | ||
|
||
**Key Changes:** | ||
- Replace data injection marker waiting logic with direct data copying. | ||
- Use your data container image. | ||
|
||
Your app should be ready to deploy. If there are any reasons that this method does not work for you, please comment in issue [#3926](https://github.com/zarf-dev/zarf/issues/3926) | ||
|
||
## Future Alternatives | ||
|
||
Once the new Kubernetes feature [OCI volume sources](https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/4639-oci-volume-source) is considered stable, this document will recommend that method. OCI volumes sources, in beta as of Kubernetes 1.33, provide a simpler way to mount data from an image into a pod. | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Was going to comment on using the
oci volume
source, glad to see it here