Skip to content
This repository was archived by the owner on Sep 20, 2025. It is now read-only.

Commit eb8dd41

Browse files
AoyuQCAoyuQC
authored
doc: add readme and gh-pages (#12)
* doc: add readme and gh-pages * per-commit fix * finish complete readme * docs: verified readme
1 parent 0891c99 commit eb8dd41

21 files changed

+220
-221
lines changed

‎.github/workflows/gh-pages.yml‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
name: GitHub Pages
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
paths:
8+
- docs/**
9+
workflow_dispatch:
10+
11+
jobs:
12+
deploy:
13+
runs-on: ubuntu-latest
14+
steps:
15+
- uses: actions/checkout@v2
16+
with:
17+
fetch-depth: 0 # Fetch all history for .GitInfo and .Lastmod
18+
19+
- name: Setup Python
20+
uses: actions/setup-python@v2
21+
with:
22+
python-version: '3.11'
23+
24+
- name: Install dependencies
25+
run: |
26+
python3 -m pip install mkdocs==1.6.1
27+
python3 -m pip install mkdocs-material==9.6.5
28+
python3 -m pip install mkdocs-include-markdown-plugin==7.1.4
29+
python3 -m pip install mkdocs-macros-plugin==1.3.7
30+
python3 -m pip install mkdocs-with-pdf==0.9.3
31+
python3 -m pip install mkdocs-print-site-plugin==2.6.0
32+
33+
- name: Build mkdocs
34+
run: |
35+
mkdocs build -f ./docs/mkdocs.en.yml
36+
cp -av ./docs/index.html ./docs/site
37+
38+
- name: Deploy
39+
uses: peaceiris/actions-gh-pages@v3
40+
if: ${{ github.ref == 'refs/heads/main' }}
41+
with:
42+
github_token: ${{ secrets.ACTION_TOKEN }}
43+
publish_dir: ./docs/site
44+
publish_branch: gh-pages

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,6 +127,7 @@ cython_debug/
127127

128128
# VS Code
129129
.vscode/
130+
docs/site
130131

131132
# data
132133
emd_models/

‎README.md‎

Lines changed: 66 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -1,64 +1,91 @@
1-
## Installation Guide
1+
<h3 align="center">
2+
Easy Model Deployer - Simple, Efficient, and Easy-to-Integrate
3+
</h3>
4+
---
25

3-
### Prerequisites
6+
*Latest News* 🔥
47

5-
Ensure you have Python 3 installed on your system. If not, download and install it from the [official Python website](https://www.python.org/downloads/).
8+
- [2025/03] We officially released EMD! Check out our [blog post](https://vllm.ai).
69

7-
### Installing pip
10+
---
811

9-
If `pip` is not installed, follow one of the methods below:
12+
## About
1013

11-
#### Method 1: Using ensurepip
14+
EMD (Easy Model Deployer) is a lightweight tool designed to simplify model deployment. Built for developers who need reliable and scalable model serving without complex setup.
1215

13-
Run the following command to install or upgrade `pip`:
16+
**Key Features**
17+
- One-click deployment of models to the cloud (Amazon SageMaker, Amazon ECS) or on-premises
18+
- Diverse model types (LLMs, VLMs, Embeddings, Vision, etc.)
19+
- Rich inference engine (vLLM, TGI, Lmdeploy, etc.)
20+
- Different instance types (CPU/GPU/AWS Inferentia)
21+
- Convenient integration (OpenAI Compatible API, LangChain client, etc.)
1422

15-
```bash
16-
python3 -m ensurepip --upgrade
17-
```
23+
**Notes**
1824

19-
#### Method 2: Using get-pip.py
25+
- Please check the [Supported Models](docs/supported_models.md) for complete list.
26+
- OpenAI Compatible API is supported only for Amazon ECS deployment.
2027

21-
Download the `get-pip.py` script and execute it:
28+
## Getting Started
2229

23-
```bash
24-
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
25-
python3 get-pip.py
26-
```
30+
### Installation
2731

28-
### Installing EMD
32+
Install EMD with `pip`, currently only support for Python 3.9 and above:
2933

30-
To install the EMD package, execute the following steps:
34+
```bash
35+
curl https://github.com/aws-samples/easy-model-deployer/releases/download/dev/emd-0.6.0-py3-none-any.whl -o emd-0.6.0-py3-none-any.whl && pip install emd-0.6.0-py3-none-any.whl"[all]"
36+
```
3137

32-
1. **(Optional)** Create and activate a virtual environment:
38+
Visit our [documentation](https://aws-samples.github.io/easy-model-deployer/) to learn more.
3339

34-
Creating a virtual environment is recommended if you want to isolate the package and its dependencies from other Python projects on your system. This helps prevent version conflicts and keeps your global Python environment clean.
40+
### Usage
3541

36-
```bash
37-
python3 -m venv .venv
38-
source .venv/bin/activate
39-
```
42+
#### Choose your default aws profile.
43+
```bash
44+
emd config set-default-profile-name
45+
```
46+
Notes: If you don't set aws profile, it will use the default profile in your env (suitable for Temporary Credentials). Whenever you want to switch deployment accounts, run ```emd config set-default-profile-name```
47+
![alt text](docs/images/emd-config.png)
4048

41-
2. **Install the package**:
49+
#### Bootstrap emd stack
50+
```bash
51+
emd bootstrap
52+
```
53+
Notes: This is going to set up the necessary resources for model deployment. Whenever you change EMD version, run this command again.
54+
![alt text](docs/images/emd-bootstrap.png)
4255

43-
```bash
44-
pip install ./
45-
```
56+
#### Choose deployment parameters interactively by ```emd deploy``` or deploy with one command
57+
```bash
58+
emd deploy --model-id DeepSeek-R1-Distill-Qwen-1.5B --instance-type g5.8xlarge --engine-type vllm --framework-type fastapi --service-type sagemaker --extra-params {} --skip-confirm
59+
```
60+
Notes: Get complete parameters by ```emd deploy --help``` and find the values of the required parameters [here](docs/en/supported_models.md)
61+
When you see "Waiting for model: ...", it means the deployment task has started, you can quit the current task by ctrl+c.
62+
![alt text](docs/images/emd-deploy.png)
4663

47-
This command installs the package along with its dependencies.
64+
#### Check deployment status.
65+
```bash
66+
emd status
67+
```
68+
![alt text](docs/images/emd-status.png)
69+
Notes: EMD allows to launch multiple deployment tasks at the same time.
4870

71+
#### Quick functional verfication or check our [documentation](https://aws-samples.github.io/easy-model-deployer/) for integration examples.
72+
```bash
73+
emd invoke DeepSeek-R1-Distill-Qwen-1.5B
74+
```
75+
Notes: Find *ModelId* in the output of ```emd status```.
76+
![alt text](docs/images/emd-invoke.png)
4977

50-
3. **(Optional) Install the package with LangChain client**:
51-
```bash
52-
pip install .[langchain_client]
53-
```
78+
#### Delete the deployed model
79+
```bash
80+
emd destroy DeepSeek-R1-Distill-Qwen-1.5B
81+
```
82+
Notes: Find *ModelId* in the output of ```emd status```.
5483

5584

56-
## Usage Instructions
85+
## Documentation
5786

58-
After successful installation, use the following command to access the CLI help:
87+
For advanced configurations and detailed guides, visit our [documentation site](https://emd-docs.example.com).
5988

60-
```bash
61-
emd --help
62-
```
89+
## Contributing
6390

64-
This command displays the available options and commands for the EMD CLI.
91+
We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

‎docs/en/about.md‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# About Easy Model Deployer
2+
3+
Easy Model Deployer is a lightweight tool designed to simplify the machine learning model deployment process.
4+
5+
## Features
6+
7+
- Simple deployment workflow
8+
- Support for multiple ML frameworks
9+
- Easy configuration
10+
- Minimal dependencies
11+
12+
## Why Use Easy Model Deployer?
13+
14+
Perfect for developers who want to quickly deploy ML models without dealing with complex infrastructure setup.
15+
16+
## Getting Started
17+
18+
Check our [Usage Guide](usage.md) to start deploying your models in minutes.

‎docs/en/installation.md‎

Whitespace-only changes.

‎docs/en/supported_models.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
| ModeId | ModelSeries | ModelType | Supported Engines | Supported Instances | Supported Services | Support China Region |
2+
|:----------------------------------|:-------------------------|:------------|:--------------------|:---------------------------------------------------------------------------------------------------------------------|:------------------------------|:-----------------------|
3+
| glm-4-9b-chat | glm4 | llm | vllm | g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ✅ |
4+
| internlm2_5-20b-chat-4bit-awq | internlm2.5 | llm | vllm | g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.12xlarge,g5.16xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ✅ |
5+
| internlm2_5-20b-chat | internlm2.5 | llm | vllm | g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ✅ |
6+
| internlm2_5-7b-chat | internlm2.5 | llm | vllm | g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.12xlarge,g5.16xlarge,g5.24xlarge,g5.48xlarge,g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ✅ |
7+
| internlm2_5-7b-chat-4bit | internlm2.5 | llm | vllm | g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.12xlarge,g5.16xlarge,g5.24xlarge,g5.48xlarge,g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ❎ |
8+
| internlm2_5-1_8b-chat | internlm2.5 | llm | vllm | g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.12xlarge,g5.16xlarge,g5.24xlarge,g5.48xlarge,g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ✅ |
9+
| Qwen2.5-7B-Instruct | qwen2.5 | llm | vllm | g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.12xlarge,g5.16xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ✅ |
10+
| Qwen2.5-72B-Instruct-AWQ | qwen2.5 | llm | vllm,tgi | g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ✅ |
11+
| Qwen2.5-72B-Instruct | qwen2.5 | llm | vllm | g5.48xlarge | sagemaker,sagemaker_async,ecs | ✅ |
12+
| Qwen2.5-72B-Instruct-AWQ-128k | qwen2.5 | llm | vllm | g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ✅ |
13+
| Qwen2.5-32B-Instruct | qwen2.5 | llm | vllm | g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ✅ |
14+
| Qwen2.5-0.5B-Instruct | qwen2.5 | llm | vllm,tgi | g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.16xlarge,inf2.8xlarge | sagemaker,sagemaker_async,ecs | ✅ |
15+
| Qwen2.5-1.5B-Instruct | qwen2.5 | llm | vllm | g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.16xlarge | sagemaker,sagemaker_async,ecs | ✅ |
16+
| Qwen2.5-3B-Instruct | qwen2.5 | llm | vllm | g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.16xlarge | sagemaker,sagemaker_async,ecs | ✅ |
17+
| Qwen2.5-14B-Instruct-AWQ | qwen2.5 | llm | vllm | g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.16xlarge | sagemaker,sagemaker_async,ecs | ✅ |
18+
| Qwen2.5-14B-Instruct | qwen2.5 | llm | vllm | g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ✅ |
19+
| QwQ-32B-Preview | qwen reasoning model | llm | huggingface,vllm | g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ✅ |
20+
| llama-3.3-70b-instruct-awq | llama | llm | tgi | g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ❎ |
21+
| DeepSeek-R1-Distill-Qwen-32B | deepseek reasoning model | llm | vllm | g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ❎ |
22+
| DeepSeek-R1-Distill-Qwen-14B | deepseek reasoning model | llm | vllm | g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ❎ |
23+
| DeepSeek-R1-Distill-Qwen-7B | deepseek reasoning model | llm | vllm | g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.16xlarge | sagemaker,sagemaker_async,ecs | ❎ |
24+
| DeepSeek-R1-Distill-Qwen-1.5B | deepseek reasoning model | llm | vllm | g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.16xlarge | sagemaker,sagemaker_async,ecs | ❎ |
25+
| DeepSeek-R1-Distill-Llama-8B | deepseek reasoning model | llm | vllm | g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.16xlarge | sagemaker,sagemaker_async,ecs | ❎ |
26+
| deepseek-r1-distill-llama-70b-awq | deepseek reasoning model | llm | tgi,vllm | g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ✅ |
27+
| Baichuan-M1-14B-Instruct | baichuan | llm | huggingface | g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async,ecs | ❎ |
28+
| Qwen2-VL-72B-Instruct-AWQ | qwen2vl | vlm | vllm | g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async | ✅ |
29+
| QVQ-72B-Preview-AWQ | qwen reasoning model | vlm | vllm | g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async | ❎ |
30+
| Qwen2-VL-7B-Instruct | qwen2vl | vlm | vllm | g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.12xlarge,g5.16xlarge,g5.24xlarge,g5.48xlarge,g6e.2xlarge | sagemaker,sagemaker_async | ✅ |
31+
| InternVL2_5-78B-AWQ | internvl2.5 | vlm | lmdeploy | g5.12xlarge,g5.24xlarge,g5.48xlarge | sagemaker,sagemaker_async | ❎ |
32+
| txt2video-LTX | comfyui | video | comfyui | g5.4xlarge,g5.8xlarge,g6e.2xlarge | sagemaker_async | ❎ |
33+
| whisper | whisper | whisper | huggingface | g5.xlarge,g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.16xlarge | sagemaker_async | ❎ |
34+
| bge-base-en-v1.5 | bge | embedding | vllm | g5.xlarge,g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.16xlarge | sagemaker | ✅ |
35+
| bge-m3 | bge | embedding | vllm | g5.xlarge,g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.16xlarge | sagemaker,ecs | ✅ |
36+
| bge-reranker-v2-m3 | bge | rerank | vllm | g5.xlarge,g5.2xlarge,g5.4xlarge,g5.8xlarge,g5.16xlarge | sagemaker | ✅ |

‎docs/en/usage.md‎

Whitespace-only changes.

‎docs/images/emd-bootstrap.png‎

524 KB
Loading

‎docs/images/emd-config.png‎

32.5 KB
Loading

‎docs/images/emd-deploy.png‎

419 KB
Loading

0 commit comments

Comments
 (0)