Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Set default behavior to automatically normalize line endings
* text=auto eol=lf

# Explicitly declare text files that should use LF
*.yaml text eol=lf
*.yml text eol=lf
*.json text eol=lf
*.js text eol=lf
*.ts text eol=lf
*.md text eol=lf
*.sh text eol=lf
*.ps1 text eol=lf
*.env text eol=lf

# Declare all files that are binary and should not be modified
*.png binary
*.jpg binary
*.jpeg binary
*.gif binary
*.ico binary
*.zip binary
*.tar binary
*.gz binary
*.db binary
*.pdf binary
*.psd binary

# Git configuration
.gitattributes text eol=lf
.gitignore text eol=lf
.gitconfig text eol=lf

# Docker configuration
Dockerfile text eol=lf
docker-compose.yml text eol=lf
docker-compose.yaml text eol=lf
29 changes: 29 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1 +1,30 @@
# n8n-hosting

This repository contains various deployment options for n8n workflow automation tool, including Docker, Docker Compose, and Kubernetes.

## Deployment Options

### Docker Compose
Simple deployment options using Docker Compose for development and small production environments.

* [Basic Setup](docker-compose/)
* [With Postgres](docker-compose/withPostgres/)
* [With Postgres and Worker](docker-compose/withPostgresAndWorker/)

### Docker with Caddy
Deployment option using Docker with Caddy as a reverse proxy.

* [Docker + Caddy](docker-caddy/)

### Kubernetes
Enterprise-grade deployment option with Kustomize overlays for production environments. Features UI/Worker separation, autoscaling via KEDA, and production-ready security hardening.

* [Kubernetes Deployment](kubernetes/)
* [Production Overlay](kubernetes/overlays/production/)
* [KEDA Autoscaling Configuration](kubernetes/configure-keda-prometheus.ps1)

## Getting Started

Choose a deployment option based on your requirements and follow the instructions in the corresponding directory.

For enterprise production deployments, the Kubernetes option offers the most scalability and resilience.
118 changes: 109 additions & 9 deletions kubernetes/README.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,123 @@
# n8n-kubernetes-hosting

Get up and running with n8n on the following platforms:
This repository provides Kubernetes manifests for deploying n8n workflow automation tool in both development and production environments. It uses Kustomize overlays to manage environment-specific configurations.

* [AWS](https://docs.n8n.io/hosting/server-setups/aws/)
* [Azure](https://docs.n8n.io/hosting/server-setups/azure/)
* [Google Cloud Platform](https://docs.n8n.io/hosting/server-setups/google-cloud/)
## Architecture Overview

If you have questions after trying the tutorials, check out the [forums](https://community.n8n.io/).
The production deployment uses a scalable architecture with the following components:

* **n8n UI**: Single pod handling the web interface (Deployment)
* **n8n Workers**: Multiple pods processing workflow executions (Deployment with KEDA autoscaling)
* **Redis**: Queue management for worker coordination (Deployment)
* **Postgres**: Persistence layer for workflow storage (Deployment)

### Key Features

* **UI/Worker Architecture**: Separates web interface and workflow processing for better scalability
* **Autoscaling**: Uses KEDA with Prometheus metrics to scale workers based on queue size
* **Security Hardening**: Non-root execution, network policies, and secret management
* **Resource Management**: Production-grade resource limits and requests
* **Zero-Downtime Updates**: RollingUpdate strategy for all components

## Prerequisites

Self-hosting n8n requires technical knowledge, including:

* Setting up and configuring servers and containers
* Kubernetes cluster administration
* Setting up and configuring containers and orchestration
* Managing application resources and scaling
* Securing servers and applications
* Configuring n8n
* Securing Kubernetes workloads
* Configuring n8n and related infrastructure

### Required Components

* Kubernetes cluster (v1.16+)
* Kubectl and Kustomize
* KEDA for autoscaling (v2.0+)
* Prometheus for metrics (optional, required for autoscaling)
* Ingress controller (e.g., nginx-ingress)
* Cert-Manager (optional, for SSL)

## Deployment Structure

The repository is organized using Kustomize overlays:

```
kubernetes/
├── base/ # Base configuration shared across environments
├── overlays/
│ └── production/ # Production-specific configurations
└── configure-keda-prometheus.ps1 # Configuration script
```

### Base Directory

Contains core components:
* n8n UI deployment (single replica)
* n8n worker deployment
* Redis and Postgres deployments
* Services, PVCs, ConfigMaps, and Secrets
* Basic ingress configuration

### Production Overlay

Enhances the base with production-ready configurations:
* Increased worker replicas (3 by default)
* Enhanced resource limits
* Redis security with password authentication
* Network policies for Redis and Postgres
* Security contexts for non-root execution
* Custom hostname through ingress-patch
* KEDA ScaledObject for worker autoscaling

## Deployment Instructions

### Basic Deployment

```bash
# Deploy the production configuration
kubectl apply -k kubernetes/overlays/production
```

### Configuring KEDA and Prometheus

The repository includes a PowerShell script that automatically detects KEDA and Prometheus in your cluster and configures the ScaledObject accordingly:

```powershell
# Run with default settings
.\kubernetes\configure-keda-prometheus.ps1

# Or customize parameters
.\kubernetes\configure-keda-prometheus.ps1 -MinReplicas 3 -MaxReplicas 30 -Threshold 10
```

#### Script Parameters

* `Namespace`: Your n8n namespace (default: "n8n")
* `PromNamespace`: Prometheus namespace (default: "prometheus")
* `ScaledObjectPatchFile`: Path to ScaledObject patch (default: "overlays/production/n8n-worker-scaledobject-patch.yaml")
* `MinReplicas`: Minimum worker replicas (default: 2)
* `MaxReplicas`: Maximum worker replicas (default: 20)
* `MetricName`: Metric name for scaling (default: "n8n_queue_waiting_jobs")
* `Threshold`: Queue size threshold to trigger scaling (default: 5)
* `Query`: Prometheus query (default: "sum(n8n_queue_waiting_jobs)")

## Environment Variables

Key environment variables in the production deployment:

* `EXECUTIONS_MODE=queue`: Enables queue mode for distributed execution
* `QUEUE_BULL_REDIS_HOST`: Points to Redis service
* `QUEUE_HEALTH_CHECK_ACTIVE=true`: Enables worker health checks
* `DB_POSTGRESDB_HOST`: Points to Postgres service
* `N8N_ENCRYPTION_KEY`: From Secret for data encryption

## Resource Allocations

n8n recommends self-hosting for expert users. Mistakes can lead to data loss, security issues, and downtime. If you aren't experienced at managing servers, n8n recommends [n8n Cloud](https://n8n.io/cloud/).
* **n8n UI**: 500Mi-1Gi memory, 500m-1 CPU
* **n8n workers**: 500Mi-1Gi memory, 300m-1 CPU
* **Redis**: 128Mi-256Mi memory, 100m-300m CPU
* **Postgres**: 4Gi-8Gi memory, 2-4 CPU

## Contributions

Expand Down
26 changes: 26 additions & 0 deletions kubernetes/base/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: n8n

labels:
- pairs:
app: n8n

resources:
- namespace.yaml
- n8n-deployment.yaml
- n8n-deployment-workers.yaml
- n8n-service.yaml
- redis-deployment.yaml
- redis-service.yaml
- pvcs.yaml
- postgres-deployment.yaml
- postgres-service.yaml
- postgres-pvc.yaml
- postgres-configmap.yaml
- postgres-secret.yaml
- n8n-secret.yaml
- n8n-ingress.yaml
- n8n-worker-scaledobject.yaml
- n8n-servicemonitor.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,22 @@ apiVersion: apps/v1
kind: Deployment
metadata:
labels:
service: n8n
name: n8n
namespace: n8n
service: n8n-worker
name: n8n-worker
spec:
replicas: 1
replicas: 2
selector:
matchLabels:
service: n8n
service: n8n-worker
strategy:
type: Recreate
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 0
template:
metadata:
labels:
service: n8n
service: n8n-worker
spec:
initContainers:
- name: volume-permissions
Expand All @@ -29,12 +31,12 @@ spec:
- /bin/sh
args:
- -c
- sleep 5; n8n start
- sleep 5; n8n worker
env:
- name: DB_TYPE
value: postgresdb
- name: DB_POSTGRESDB_HOST
value: postgres-service.n8n.svc.cluster.local
value: postgres-service
- name: DB_POSTGRESDB_PORT
value: "5432"
- name: DB_POSTGRESDB_DATABASE
Expand All @@ -43,25 +45,41 @@ spec:
valueFrom:
secretKeyRef:
name: postgres-secret
key: POSTGRES_NON_ROOT_USER
key: POSTGRES_USER
- name: DB_POSTGRESDB_PASSWORD
valueFrom:
secretKeyRef:
name: postgres-secret
key: POSTGRES_NON_ROOT_PASSWORD
- name: N8N_PROTOCOL
value: http
- name: N8N_PORT
value: "5678"
key: POSTGRES_PASSWORD
- name: EXECUTIONS_MODE
value: queue
- name: QUEUE_BULL_REDIS_HOST
value: redis-service
- name: QUEUE_HEALTH_CHECK_ACTIVE
value: "true"
- name: N8N_ENCRYPTION_KEY
valueFrom:
secretKeyRef:
name: n8n-secret
key: N8N_ENCRYPTION_KEY
- name: N8N_METRICS
value: "true"
- name: N8N_METRICS_PORT
value: "9100"
- name: N8N_METRICS_PREFIX
value: "n8n_"
image: n8nio/n8n
name: n8n
name: n8n-worker
ports:
- containerPort: 5678
- containerPort: 9100
name: metrics
resources:
requests:
memory: "250Mi"
cpu: "100m"
limits:
memory: "500Mi"
cpu: "500m"
volumeMounts:
- mountPath: /home/node/.n8n
name: n8n-claim0
Expand Down
Loading