Big Data Project

Theme

IMDB Movies & Series reviews processed as batch & Rotten Tomatoes reviews processed as streaming

Technologies

• Spark
• Spark Streaming
• Kafka
• Elasticsearch
• Kibana
• Python (for Spark interaction)
• HDFS
• Docker

Prerequisites

• Docker
• WSL/macOS/Linux

How to run

cd docker
./run.sh

Open Kibana Dashboard: localhost:5601

Diagram