Skip to content
This repository was archived by the owner on Sep 20, 2025. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 12 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,39 +22,27 @@

## Introduction

Easy Model Deployer is a lightweight tool designed to simplify the deployment of **Open-Source LLMs** ([Supported Models](docs/en/supported_models.md)) and Custom Models. It provides **OpenAI's Completions API** and [**LangChain Interface**](https://github.com/langchain-ai/langchain). Built for developers who need reliable and scalable model serving without complex setup, it seamlessly integrates with AWS services for efficient model deployment.
Easy Model Deployer is a lightweight tool designed for simplify deploy **Open-Source LLMs** ([Supported Models](docs/en/supported_models.md)) and Custom Models on AWS. It provides **OpenAI's Completions API** and [**LangChain Interface**](https://github.com/langchain-ai/langchain). Built for developers who need reliable and scalable model serving without complex environment setup.

![cli](docs/images/cli-all.gif)
![cli](docs/images/demo.avif)

**Key Features**

- One-click deployment of models to the cloud (Amazon SageMaker, Amazon ECS, Amazon EC2)
- One-click deployment of models to AWS (Amazon SageMaker, Amazon ECS, Amazon EC2)
- Diverse model types (LLMs, VLMs, Embeddings, Vision, etc.)
- Rich inference engine (vLLM, TGI, Lmdeploy, etc.)
- Different instance types (CPU/GPU/AWS Inferentia)
- Convenient integration (OpenAI Compatible API, LangChain client, etc.)

## 🚀 Quick Install
## 🔧 Get Started

Install Easy Model Deployer with `pip`, currently support for Python 3.9 and above:
### Installation

```bash
pip install easy_model_deployer # or pip3 install easy_model_deployer
emd
```

Visit our [documentation](https://aws-samples.github.io/easy-model-deployer/en/installation/) to learn more.

## 🔧 Usage

### (Optional) Configure AWS Profile

> If you have already configured your AWS credentials using the AWS CLI, you can skip this step.

You can configure the AWS profile by EMD. If you don't set any AWS profile, the EMD will use the default credentials in your terminal, you can also configure your credentials by [`aws configure`](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html#cli-configure-files-methods).
Install Easy Model Deployer with PyPI, currently support for Python 3.9 or above:

```bash
emd config set-default-profile-name
pip install easy-model-deployer
emd
```

### Bootstrap
Expand All @@ -67,18 +55,16 @@ For more information, please refer to [Architecture](https://aws-samples.github.
emd bootstrap
```

> **💡 Tip** Once you upgrade the EMD by `pip`, you need to run this command again.
> **💡 Tip** Once you upgrade the EMD by `pip`, you need to run this command again to update the environment.

### Deploy Model
### Deploy Models

Deploy models with an interactive CLI or one command.
Deploy models with an interactive CLI or one command line.

```bash
emd deploy
```

![deploy](docs/images/cli-deploy.gif)

> **💡 Tip** To view all available parameters, run `emd deploy --help`.
> When you see the message "Waiting for model: ...", it means the deployment task has started and you can stop the terminal output by pressing `Ctrl+C`.
>
Expand All @@ -95,7 +81,7 @@ emd status

> **💡 Tip** The EMD allows launch multiple deployment tasks simultaneously.

### Quick Invocation
### Invocation

Invoke the deployed model for testing by CLI.

Expand All @@ -120,11 +106,6 @@ Quickly see what models are supported, this command will output all information
emd list-supported-models
```

The following command is recommended to just list the model types.

```bash
emd list-supported-models | jq -r '.[] | "\(.model_id)\t\(.model_type)"' | column -t -s $'\t' | sort
```

### Delete Model

Expand Down
Binary file removed docs/images/cli-all.gif
Binary file not shown.
Binary file removed docs/images/cli-deploy.gif
Binary file not shown.
Binary file removed docs/images/cli.gif
Binary file not shown.
Binary file added docs/images/demo.avif
Binary file not shown.
Binary file removed docs/images/emd-bootstrap.png
Binary file not shown.
Binary file removed docs/images/emd-config.png
Binary file not shown.
Binary file removed docs/images/emd-deploy.png
Binary file not shown.
Binary file removed docs/images/emd-destroy.png
Binary file not shown.
Binary file removed docs/images/emd-invoke.png
Binary file not shown.
Binary file removed docs/images/emd-status.png
Binary file not shown.