ASL_Detector is a LSTM model that detects American Sign Language (ASL) gestures and translates them into English using Gemini or Nebius API.
bash
py -3.12 -m venv venv312
(
Or create .venv outside of folder with the help of VSC IDE:
- Navigate to .ipynb file
- On the top right corner, there's a kernel box that displays either your current Python version or the text: "Select Kernel"
- Click on the kernel box, click on "Select Another Kernel"
- Click on "Python Environments..."
- Click on "Create Python Environment"
- Click on Venv (.venv), choose the Python version that's compatible (3.9 - 3.12)
)
.\venv312\Scripts\activate
(or ..\.venv\Scripts\activate)
pip install -r requirements.txt
python3.11 -m venv venv311
source venv311/bin/activate
pip install -r requirements.txt
Delete cv2.CAP_DSHOW to prevent problems
The main notebook in this project is: asl_alphabet_new.ipynb
It contains 4 main sections:
- Collecting Data – Capture ASL hand signs using your webcam.
- Create Dataset – Preprocess and structure the collected data.
- Train Model – Train a Convolutional Neural Network (CNN) using PyTorch.
- Live Test – Use your webcam to test the trained model in real-time.
If you want to use the translation part, remember to .env file with your API KEY. It should look like this GEMINI_API_KEY=<your_key>
Run new_live_test_3D.py to try the translation from ASL to English.
-
Activate virtual environment
-
Recording video with realtime ASL recognition, then annotate (with both raw and result videos saved afterwards):
py .\live_save_test.py --realtime
-
Annotating existing video:
python live_save_test.py --input raw_videos/test2.mp4 --output_dir ./annotated_videos -
Recording video without realtime ASL recognition, then annotate (with both raw and result videos saved afterwards):
py .\live_save_test.py --record
Raw / Input videos will / must be stored in raw_videos
Result videos will be generated in annotated_videos