From 1d3e177e27e098561d097ebf95442c198d4538d7 Mon Sep 17 00:00:00 2001 From: sgolebiewski-intel Date: Thu, 30 Oct 2025 13:11:18 +0100 Subject: [PATCH] [DOCS] Update Metro SDK docs for publishing --- metro-ai-suite/metro-sdk-manager/docs/toc.rst | 3 - .../docs/user-guide/{index.rst => index.md} | 54 ++---- .../docs/user-guide/install-sdk.md | 2 +- .../metro-gen-ai-sdk/get-started.md | 30 ++- .../metro-vision-ai-sdk/get-started.md | 54 ++++-- .../metro-vision-ai-sdk/tutorial-1.md | 3 +- .../metro-vision-ai-sdk/tutorial-2.md | 18 +- .../metro-vision-ai-sdk/tutorial-3.md | 183 +++++++++--------- .../metro-vision-ai-sdk/tutorial-4.md | 2 + .../metro-vision-ai-sdk/tutorial-5.md | 20 +- .../docs/user-guide/support.md | 1 + .../visual-ai-demo-kit/get-started.md | 46 ++++- .../visual-ai-demo-kit/tutorial-1.md | 17 +- .../visual-ai-demo-kit/tutorial-2.md | 24 ++- .../visual-ai-demo-kit/tutorial-3.md | 11 +- 15 files changed, 272 insertions(+), 196 deletions(-) delete mode 100644 metro-ai-suite/metro-sdk-manager/docs/toc.rst rename metro-ai-suite/metro-sdk-manager/docs/user-guide/{index.rst => index.md} (57%) diff --git a/metro-ai-suite/metro-sdk-manager/docs/toc.rst b/metro-ai-suite/metro-sdk-manager/docs/toc.rst deleted file mode 100644 index 67928b90b..000000000 --- a/metro-ai-suite/metro-sdk-manager/docs/toc.rst +++ /dev/null @@ -1,3 +0,0 @@ -.. toctree:: - - user-guide/index \ No newline at end of file diff --git a/metro-ai-suite/metro-sdk-manager/docs/user-guide/index.rst b/metro-ai-suite/metro-sdk-manager/docs/user-guide/index.md similarity index 57% rename from metro-ai-suite/metro-sdk-manager/docs/user-guide/index.rst rename to metro-ai-suite/metro-sdk-manager/docs/user-guide/index.md index 14e8afb0e..17cbc514e 100644 --- a/metro-ai-suite/metro-sdk-manager/docs/user-guide/index.rst +++ b/metro-ai-suite/metro-sdk-manager/docs/user-guide/index.md @@ -1,47 +1,19 @@ -Metro SDK Manager -============================================ +# Metro SDK Manager - -Overview --------- +## Overview The SDK Manager is a comprehensive development tool that streamlines the process of discovering, installing, and managing multiple software development kits for edge AI applications. Designed for developers working with Intel's edge computing ecosystem, it eliminates the complexity of manually configuring SDK combinations by providing automated dependency resolution, version compatibility checking, and integrated toolchain setup. The tool supports interactive wizard-guided selection for developers to download and install the SDK directly into their development environment. Built with developer productivity in mind, the SDK Manager handles cross-platform builds, maintains isolated development environments, and provides real-time compatibility validation across different SDK versions. It includes extensive documentation with code samples, API references, and troubleshooting guides. Whether you're prototyping edge AI solutions or deploying production applications, the SDK Manager ensures consistent, reproducible development environments across your team and deployment targets. -.. toctree:: - :caption: Metro SDK Manager - - install-sdk - -.. toctree:: - :caption: Metro Vision AI SDK - :hidden: - - metro-vision-ai-sdk/get-started.md - metro-vision-ai-sdk/tutorial-1.md - metro-vision-ai-sdk/tutorial-2.md - metro-vision-ai-sdk/tutorial-3.md - metro-vision-ai-sdk/tutorial-4.md - metro-vision-ai-sdk/tutorial-5.md - -.. toctree:: - :caption: Metro Gen AI SDK - :hidden: - - metro-gen-ai-sdk/get-started.md - -.. toctree:: - :caption: Visual AI Demo Kit - :hidden: - - visual-ai-demo-kit/get-started.md - visual-ai-demo-kit/tutorial-1.md - visual-ai-demo-kit/tutorial-2.md - visual-ai-demo-kit/tutorial-3.md - -.. toctree:: - :caption: Community and Support - :hidden: - - support + diff --git a/metro-ai-suite/metro-sdk-manager/docs/user-guide/install-sdk.md b/metro-ai-suite/metro-sdk-manager/docs/user-guide/install-sdk.md index a283b887b..9d0f6adda 100644 --- a/metro-ai-suite/metro-sdk-manager/docs/user-guide/install-sdk.md +++ b/metro-ai-suite/metro-sdk-manager/docs/user-guide/install-sdk.md @@ -2,4 +2,4 @@ - + diff --git a/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-gen-ai-sdk/get-started.md b/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-gen-ai-sdk/get-started.md index 1553e1b50..506159bd2 100644 --- a/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-gen-ai-sdk/get-started.md +++ b/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-gen-ai-sdk/get-started.md @@ -7,6 +7,7 @@ The Metro Gen AI SDK provides a comprehensive development environment for genera ## Learning Objectives Upon completion of this guide, you will be able to: + - Install and configure the Metro Gen AI SDK - Deploy generative AI microservices for document processing and question-answering - Understand the architecture of RAG-based applications using Intel's AI frameworks @@ -30,7 +31,6 @@ curl https://raw.githubusercontent.com/open-edge-platform/edge-ai-suites/refs/he ![Metro Gen AI SDK Installation](images/metro-gen-ai-sdk-install.png) - ## Question-Answering Application Implementation This section demonstrates a complete RAG (Retrieval-Augmented Generation) application workflow using the installed Gen AI components. @@ -69,6 +69,7 @@ Start the complete Gen AI application stack using Docker Compose: ```bash docker compose up ``` + ### Step 4: Verify Deployment Status Check that all application components are running correctly: @@ -77,7 +78,6 @@ Check that all application components are running correctly: docker ps ``` - ### Step 5: Access the Application Interface Open a web browser and navigate to the application dashboard: @@ -89,13 +89,23 @@ http://localhost:8101 ## Additional Resources ### Technical Documentation -- [Audio Analyzer](https://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/audio-analyzer/index.html) - Comprehensive documentation for multimodal audio processing capabilities -- [Document Ingestion - pgvector](https://github.com/open-edge-platform/edge-ai-libraries/blob/main/microservices/document-ingestion/pgvector/docs/get-started.md) - Vector database integration and document processing workflows -- [Multimodal Embedding Serving](https://github.com/open-edge-platform/edge-ai-libraries/blob/main/microservices/multimodal-embedding-serving/docs/user-guide/Overview.md) - Embedding generation service architecture and API documentation -- [Visual Data Preparation For Retrieval](https://github.com/open-edge-platform/edge-ai-libraries/blob/main/microservices/visual-data-preparation-for-retrieval/vdms/docs/user-guide/Overview.md) - VDMS integration and visual data management workflows -- [VLM OpenVINO Serving](https://github.com/open-edge-platform/edge-ai-libraries/blob/main/microservices/vlm-openvino-serving/docs/user-guide/Overview.md) - Vision-language model deployment and optimization guidelines -- [Edge AI Libraries](https://docs.openedgeplatform.intel.com/dev/ai-libraries.html) - Complete development toolkit documentation and microservice API references -- [Edge AI Suites](https://docs.openedgeplatform.intel.com/dev/ai-suite-metro.html) - Comprehensive application suite documentation with Gen AI implementation examples + +- [Audio Analyzer](https://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/audio-analyzer/index.html) + \- Comprehensive documentation for multimodal audio processing capabilities +- [Document Ingestion - pgvector](https://github.com/open-edge-platform/edge-ai-libraries/blob/main/microservices/document-ingestion/pgvector/docs/get-started.md) + \- Vector database integration and document processing workflows +- [Multimodal Embedding Serving](https://github.com/open-edge-platform/edge-ai-libraries/blob/main/microservices/multimodal-embedding-serving/docs/user-guide/Overview.md) + \- Embedding generation service architecture and API documentation +- [Visual Data Preparation For Retrieval](https://github.com/open-edge-platform/edge-ai-libraries/blob/main/microservices/visual-data-preparation-for-retrieval/vdms/docs/user-guide/Overview.md) + \- VDMS integration and visual data management workflows +- [VLM OpenVINO Serving](https://github.com/open-edge-platform/edge-ai-libraries/blob/main/microservices/vlm-openvino-serving/docs/user-guide/Overview.md) + \- Vision-language model deployment and optimization guidelines +- [Edge AI Libraries](https://docs.openedgeplatform.intel.com/dev/ai-libraries.html) + \- Complete development toolkit documentation and microservice API references +- [Edge AI Suites](https://docs.openedgeplatform.intel.com/dev/ai-suite-metro.html) + \- Comprehensive application suite documentation with Gen AI implementation examples ### Support Channels -- [GitHub Issues](https://github.com/open-edge-platform/edge-ai-libraries/issues) - Technical issue tracking and community support for Gen AI applications \ No newline at end of file + +- [GitHub Issues](https://github.com/open-edge-platform/edge-ai-libraries/issues) + \- Technical issue tracking and community support for Gen AI applications \ No newline at end of file diff --git a/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/get-started.md b/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/get-started.md index a9e09d35e..1cca20e3e 100644 --- a/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/get-started.md +++ b/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/get-started.md @@ -7,6 +7,7 @@ The Metro Vision AI SDK provides a comprehensive development environment for com ## Learning Objectives Upon completion of this guide, you will be able to: + - Install and configure the Metro Vision AI SDK - Execute object detection inference on video content - Understand the basic pipeline architecture for computer vision workflows @@ -31,6 +32,7 @@ curl https://raw.githubusercontent.com/open-edge-platform/edge-ai-suites/refs/he ![Metro Vision AI SDK Installation](images/metro-vision-ai-sdk-install.png) The installation process configures the following components: + - Docker containerization platform - Intel DLStreamer video analytics framework - OpenVINO inference optimization toolkit @@ -96,14 +98,18 @@ The resulting output displays the original video content with overlaid detection ## Technology Framework Overview ### DLStreamer Framework + DLStreamer provides a comprehensive video analytics framework built on GStreamer technology. Key capabilities include: + - Multi-format video input support (files, network streams, camera devices) - Real-time inference execution on video frame sequences - Flexible output rendering and storage options - Modular pipeline architecture for custom workflow development ### OpenVINO Optimization Toolkit + OpenVINO delivers cross-platform inference optimization for Intel hardware architectures. Core features include: + - Model format standardization through Intermediate Representation (IR) - Hardware-specific performance optimization - Extensive pre-trained model repository @@ -113,28 +119,52 @@ OpenVINO delivers cross-platform inference optimization for Intel hardware archi Continue your learning journey with these hands-on tutorials: -### [Tutorial 1: OpenVINO Model Benchmark](tutorial-1.md) +### [Tutorial 1: OpenVINO Model Benchmark](./tutorial-1.md) + Learn to benchmark AI model performance across different Intel hardware (CPU, GPU, NPU) and understand optimization techniques for production deployments. -### [Tutorial 2: Multi-Stream Video Processing](tutorial-2.md) +### [Tutorial 2: Multi-Stream Video Processing](./tutorial-2.md) + Build scalable video analytics solutions by processing multiple video streams simultaneously with hardware acceleration. -### [Tutorial 3: Real-Time Object Detection](tutorial-3.md) +### [Tutorial 3: Real-Time Object Detection](./tutorial-3.md) + Develop a complete object detection application with interactive controls, performance monitoring, and production-ready features. -### [Tutorial 4: Advanced Video Analytics Pipelines](tutorial-4.md) -Create sophisticated video analytics using Intel® DL Streamer framework, including human pose estimation and multi-model integration. +### [Tutorial 4: Advanced Video Analytics Pipelines](./tutorial-4.md) +Create sophisticated video analytics using Intel® DL Streamer framework, including human pose estimation and multi-model integration. ## Additional Resources ### Technical Documentation -- [DLStreamer](http://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/dl-streamer/index.html) - Comprehensive documentation for Intel's GStreamer-based video analytics framework -- [DLStreamer Pipeline Server](https://docs.openedgeplatform.intel.com/edge-ai-libraries/dlstreamer-pipeline-server/main/user-guide/Overview.html) - RESTful microservice architecture documentation for scalable video analytics deployment -- [OpenVINO](https://docs.openvino.ai/2025/get-started.html) - Complete reference for Intel's cross-platform inference optimization toolkit -- [OpenVINO Model Server](https://docs.openvino.ai/2025/model-server/ovms_what_is_openvino_model_server.html) - Model serving infrastructure documentation for production deployments -- [Edge AI Libraries](https://docs.openedgeplatform.intel.com/dev/ai-libraries.html) - Comprehensive development toolkit documentation and API references -- [Edge AI Suites](https://docs.openedgeplatform.intel.com/dev/ai-suite-metro.html) - Complete application suite documentation with implementation examples + +- [DLStreamer](http://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/dl-streamer/index.html) + \- Comprehensive documentation for Intel's GStreamer-based video analytics framework +- [DLStreamer Pipeline Server](https://docs.openedgeplatform.intel.com/edge-ai-libraries/dlstreamer-pipeline-server/main/user-guide/Overview.html) + \- RESTful microservice architecture documentation for scalable video analytics deployment +- [OpenVINO](https://docs.openvino.ai/2025/get-started.html) + \- Complete reference for Intel's cross-platform inference optimization toolkit +- [OpenVINO Model Server](https://docs.openvino.ai/2025/model-server/ovms_what_is_openvino_model_server.html) + \- Model serving infrastructure documentation for production deployments +- [Edge AI Libraries](https://docs.openedgeplatform.intel.com/dev/ai-libraries.html) + \- Comprehensive development toolkit documentation and API references +- [Edge AI Suites](https://docs.openedgeplatform.intel.com/dev/ai-suite-metro.html) + \- Complete application suite documentation with implementation examples ### Support Channels -- [GitHub Issues](https://github.com/open-edge-platform/edge-ai-suites/issues) - Technical issue tracking and community support \ No newline at end of file + +- [GitHub Issues](https://github.com/open-edge-platform/edge-ai-suites/issues) + \- Technical issue tracking and community support + + \ No newline at end of file diff --git a/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-1.md b/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-1.md index 70432c839..5a86f73b9 100644 --- a/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-1.md +++ b/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-1.md @@ -65,6 +65,7 @@ docker run --rm --user=root \ ``` This command will: + - Download the YOLOv10s object detection model - Convert it to OpenVINO IR format (FP16 precision) - Store the model files in the `public/yolov10s/FP16/` directory @@ -123,4 +124,4 @@ docker run -it --rm \ -m /home/openvino/public/yolov10s/FP16/yolov10s.xml \ -i /home/openvino/bottle-detection.mp4 \ -d NPU -``` \ No newline at end of file +``` diff --git a/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-2.md b/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-2.md index a08e81a03..85f4f1e87 100644 --- a/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-2.md +++ b/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-2.md @@ -6,7 +6,7 @@ This tutorial demonstrates advanced video processing capabilities using Intel's Multi-stream video processing is essential for applications like video surveillance, broadcasting, and media production. This tutorial showcases how Intel's hardware acceleration can efficiently decode and composite 16 simultaneous video streams into a single 4K display output, demonstrating the power of Intel® Quick Sync Video technology. -> ** Platform Compatibility** +> **Platform Compatibility** > This tutorial requires Intel® Core™ or Intel® Core™ Ultra processors with integrated graphics. Intel® Xeon® processors without integrated graphics are not supported for this specific use case. ## Time to Complete @@ -61,6 +61,7 @@ wget -O videos/Big_Buck_Bunny.mp4 "https://test-videos.co.uk/vids/bigbuckbunny/m **Alternative Download Method:** If the above link doesn't work, you can download from the official source: + ```bash # Alternative: Download from Internet Archive wget -O videos/Big_Buck_Bunny.mp4 "https://archive.org/download/BigBuckBunny_124/Content/big_buck_bunny_720p_surround.mp4" @@ -138,6 +139,7 @@ EOF The script creates a complex pipeline with these key components: **Pipeline Architecture:** + - **Input Sources**: 16 identical video file streams - **Decoder**: `vah264dec` - Hardware-accelerated H.264 decoding using VAAPI - **Scaling**: `vapostproc` - Hardware-accelerated video post-processing and scaling @@ -145,7 +147,8 @@ The script creates a complex pipeline with these key components: - **Output**: `xvimagesink` - X11-based video display **Tiled Layout Configuration:** -``` + +```text ┌─────────┬─────────┬─────────┬─────────┐ │ Stream1 │ Stream2 │ Stream3 │ Stream4 │ ← Row 1 (y=0) │ 0,0 │ 960,0 │1920,0 │2880,0 │ @@ -162,6 +165,7 @@ The script creates a complex pipeline with these key components: ``` **Performance Optimizations:** + - **VAAPI Acceleration**: Hardware-accelerated decoding, scaling, and composition - **Fast Scaling**: `scale-method=fast` for optimal performance - **Async Display**: `sync=false` to prevent frame dropping @@ -229,7 +233,6 @@ htop cat /sys/class/drm/card0/gt/gt0/rc6_residency_ms ``` - ### Step 6: Stop the Application To stop the video processing pipeline: @@ -251,7 +254,6 @@ xhost -local:docker docker system prune -f ``` - ## Understanding the Technology ### Intel® Quick Sync Video Technology @@ -259,14 +261,16 @@ docker system prune -f This tutorial leverages Intel's hardware-accelerated video processing capabilities: **Hardware Acceleration Benefits:** + - **Dedicated Video Engines**: Separate silicon for video decode/encode operations -- **CPU Offloading**: Frees CPU resources for other computational tasks +- **CPU Offloading**: Frees CPU resources for other computational tasks - **Power Efficiency**: Lower power consumption compared to software decoding - **Parallel Processing**: Multiple decode engines can process streams simultaneously ### VAAPI Integration **Video Acceleration API (VAAPI)** provides: + - **Hardware Abstraction**: Unified interface across Intel graphics generations - **Pipeline Optimization**: Direct GPU memory access without CPU copies - **Format Support**: Hardware acceleration for H.264, H.265, VP9, and AV1 codecs @@ -277,14 +281,16 @@ This tutorial leverages Intel's hardware-accelerated video processing capabiliti The tutorial demonstrates advanced GStreamer concepts: **Element Types:** + - **Source Elements**: `filesrc` - File input -- **Demuxer Elements**: `qtdemux` - Container format parsing +- **Demuxer Elements**: `qtdemux` - Container format parsing - **Decoder Elements**: `vah264dec` - Hardware-accelerated decoding - **Transform Elements**: `vapostproc` - Hardware scaling and format conversion - **Compositor Elements**: `vacompositor` - Multi-stream composition - **Sink Elements**: `xvimagesink` - Display output **Pipeline Benefits:** + - **Zero-Copy Operations**: Direct GPU memory transfers - **Parallel Processing**: Concurrent decode of multiple streams - **Dynamic Reconfiguration**: Runtime pipeline modifications diff --git a/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-3.md b/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-3.md index d0a562377..671199592 100644 --- a/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-3.md +++ b/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-3.md @@ -58,7 +58,6 @@ wget -O bottle-detection.mp4 https://storage.openvinotoolkit.org/test_data/video ``` - ### Step 2: Download and Prepare YOLOv10s Model Download the YOLOv10s object detection model and convert it to OpenVINO format: @@ -73,12 +72,12 @@ docker run --rm --user=root \ ``` **Model Download Process:** + 1. Downloads YOLOv10s PyTorch model from official repository 2. Converts to OpenVINO Intermediate Representation (IR) format 3. Optimizes for Intel hardware acceleration 4. Saves model files in FP16 precision for optimal performance - ### Step 3: Create Real-Time Object Detection Application Create a comprehensive Python application for real-time object detection with advanced features: @@ -103,7 +102,7 @@ from openvino.runtime import Core # COCO dataset class names (80 classes) COCO_CLASSES = [ - "person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", "truck", + "person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", "truck", "boat", "traffic light", "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee", @@ -113,17 +112,17 @@ COCO_CLASSES = [ "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "couch", "potted plant", "bed", "dining table", "toilet", "tv", "laptop", "mouse", "remote", "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", - "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", + "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", "hair drier", "toothbrush" ] class YOLOv10Detector: """Advanced YOLOv10s object detection with OpenVINO optimization""" - + def __init__(self, model_path, device="CPU", conf_threshold=0.4, iou_threshold=0.45): """ Initialize YOLOv10s detector - + Args: model_path (str): Path to YOLOv10s OpenVINO model (.xml) device (str): OpenVINO device (CPU, GPU, NPU) @@ -134,187 +133,187 @@ class YOLOv10Detector: self.device = device self.conf_threshold = conf_threshold self.iou_threshold = iou_threshold - + # Performance monitoring self.fps_list = [] self.frame_count = 0 self.total_inference_time = 0 - + # Initialize OpenVINO self._init_openvino() - + def _init_openvino(self): """Initialize OpenVINO Runtime and compile model""" try: print(f"[INFO] Initializing OpenVINO Runtime...") self.core = Core() - + # Load and compile model print(f"[INFO] Loading model: {self.model_path}") model = self.core.read_model(self.model_path) self.compiled_model = self.core.compile_model(model, self.device) - + # Get input/output information self.input_layer = self.compiled_model.input(0) self.output_layer = self.compiled_model.output(0) - + # Model input shape: [N, C, H, W] self.input_shape = self.input_layer.shape self.input_height = self.input_shape[2] self.input_width = self.input_shape[3] - + print(f"[INFO] Model loaded successfully on {self.device}") print(f"[INFO] Input shape: {self.input_shape}") print(f"[INFO] Expected input size: {self.input_width}x{self.input_height}") - + except Exception as e: print(f"[ERROR] Failed to initialize OpenVINO: {e}") sys.exit(1) - + def preprocess(self, frame): """ Preprocess input frame for YOLOv10s inference - + Args: frame: Input BGR image from OpenCV - + Returns: Preprocessed image tensor ready for inference """ # Resize to model input size resized = cv2.resize(frame, (self.input_width, self.input_height)) - + # Convert BGR to RGB rgb_image = cv2.cvtColor(resized, cv2.COLOR_BGR2RGB) - + # Normalize to [0, 1] and convert to float32 normalized = rgb_image.astype(np.float32) / 255.0 - + # Transpose to CHW format and add batch dimension input_tensor = np.transpose(normalized, (2, 0, 1)) input_tensor = np.expand_dims(input_tensor, axis=0) - + return input_tensor - + def postprocess(self, predictions, frame_shape): """ Postprocess YOLOv10s predictions to extract bounding boxes - + Args: predictions: Raw model output frame_shape: Original frame dimensions (H, W, C) - + Returns: Lists of boxes, confidences, and class IDs """ predictions = np.squeeze(predictions) - + boxes, confidences, class_ids = [], [], [] - + # Original frame dimensions orig_height, orig_width = frame_shape[:2] - + # Scale factors for coordinate conversion x_scale = orig_width / self.input_width y_scale = orig_height / self.input_height - + for detection in predictions: # YOLOv10 output format: [cx, cy, w, h, objectness, class_scores...] center_x, center_y, width, height = detection[:4] objectness = detection[4] - + # Filter by objectness confidence if objectness < self.conf_threshold: continue - + # Get class scores and find best class class_scores = detection[5:] class_id = np.argmax(class_scores) class_confidence = class_scores[class_id] - + # Combined confidence score total_confidence = objectness * class_confidence - + if total_confidence > self.conf_threshold: # Convert center coordinates to corner coordinates x1 = int((center_x - width / 2) * x_scale) y1 = int((center_y - height / 2) * y_scale) w = int(width * x_scale) h = int(height * y_scale) - + # Ensure coordinates are within frame bounds x1 = max(0, x1) y1 = max(0, y1) w = min(w, orig_width - x1) h = min(h, orig_height - y1) - + boxes.append([x1, y1, w, h]) confidences.append(float(total_confidence)) class_ids.append(int(class_id)) - + # Apply Non-Maximum Suppression if boxes: - indices = cv2.dnn.NMSBoxes(boxes, confidences, + indices = cv2.dnn.NMSBoxes(boxes, confidences, self.conf_threshold, self.iou_threshold) if len(indices) > 0: indices = indices.flatten() - return ([boxes[i] for i in indices], + return ([boxes[i] for i in indices], [confidences[i] for i in indices], [class_ids[i] for i in indices]) - + return [], [], [] - + def draw_detections(self, frame, boxes, confidences, class_ids): """ Draw bounding boxes and labels on frame - + Args: frame: Input frame to draw on boxes: List of bounding boxes [x, y, w, h] confidences: List of confidence scores class_ids: List of class IDs - + Returns: Frame with drawn detections """ for box, confidence, class_id in zip(boxes, confidences, class_ids): x, y, w, h = box - + # Get class name class_name = COCO_CLASSES[class_id] if class_id < len(COCO_CLASSES) else f"Class_{class_id}" - + # Choose color based on class color = self._get_color(class_id) - + # Draw bounding box cv2.rectangle(frame, (x, y), (x + w, y + h), color, 2) - + # Prepare label text label = f"{class_name}: {confidence:.2f}" - + # Calculate text size for background (text_width, text_height), baseline = cv2.getTextSize( label, cv2.FONT_HERSHEY_SIMPLEX, 0.5, 1) - + # Draw label background - cv2.rectangle(frame, (x, y - text_height - baseline), + cv2.rectangle(frame, (x, y - text_height - baseline), (x + text_width, y), color, -1) - + # Draw label text cv2.putText(frame, label, (x, y - baseline), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 1) - + return frame - + def _get_color(self, class_id): """Generate consistent color for each class""" np.random.seed(class_id) return tuple(np.random.randint(0, 255, 3).tolist()) - + def run_inference(self, video_path, save_output=None): """ Run real-time object detection on video - + Args: video_path (str): Path to input video file save_output (str): Optional path to save output video @@ -324,69 +323,69 @@ class YOLOv10Detector: if not cap.isOpened(): print(f"[ERROR] Unable to open video: {video_path}") return - + # Get video properties fps = int(cap.get(cv2.CAP_PROP_FPS)) width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT)) - + print(f"[INFO] Video properties: {width}x{height} @ {fps}FPS, {total_frames} frames") - + # Setup video writer if saving output writer = None if save_output: fourcc = cv2.VideoWriter_fourcc(*'mp4v') writer = cv2.VideoWriter(save_output, fourcc, fps, (width, height)) print(f"[INFO] Saving output to: {save_output}") - + print(f"[INFO] Starting object detection inference...") print(f"[INFO] Device: {self.device}") print(f"[INFO] Press 'q' to quit, 's' to save frame, 'p' to pause") - + paused = False - + while True: if not paused: ret, frame = cap.read() if not ret: print("[INFO] End of video reached") break - + self.frame_count += 1 - + # Measure inference time start_time = time.time() - + # Preprocess input_tensor = self.preprocess(frame) - + # Run inference predictions = self.compiled_model([input_tensor])[self.output_layer] - + # Postprocess boxes, confidences, class_ids = self.postprocess(predictions, frame.shape) - + inference_time = time.time() - start_time self.total_inference_time += inference_time - + # Calculate FPS current_fps = 1.0 / inference_time if inference_time > 0 else 0 self.fps_list.append(current_fps) - + # Draw detections result_frame = self.draw_detections(frame, boxes, confidences, class_ids) - + # Draw performance information self._draw_performance_info(result_frame, current_fps, len(boxes)) - + # Save frame if requested if writer: writer.write(result_frame) - + # Display result cv2.imshow("YOLOv10s Real-Time Object Detection", result_frame) - + # Handle keyboard input key = cv2.waitKey(1) & 0xFF if key == ord('q'): @@ -400,34 +399,34 @@ class YOLOv10Detector: elif key == ord('p'): paused = not paused print(f"[INFO] Playback {'paused' if paused else 'resumed'}") - + # Cleanup cap.release() if writer: writer.release() cv2.destroyAllWindows() - + # Print performance statistics self._print_statistics() - + def _draw_performance_info(self, frame, fps, detection_count): """Draw performance information on frame""" # FPS display cv2.putText(frame, f"FPS: {fps:.1f}", (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 255, 0), 2) - + # Detection count cv2.putText(frame, f"Detections: {detection_count}", (10, 60), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 255, 0), 2) - + # Device info cv2.putText(frame, f"Device: {self.device}", (10, 90), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 255, 0), 2) - + # Frame counter cv2.putText(frame, f"Frame: {self.frame_count}", (10, 120), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 255, 0), 2) - + def _print_statistics(self): """Print comprehensive performance statistics""" if self.fps_list: @@ -435,7 +434,7 @@ class YOLOv10Detector: min_fps = np.min(self.fps_list) max_fps = np.max(self.fps_list) std_fps = np.std(self.fps_list) - + print("\n" + "="*60) print("PERFORMANCE STATISTICS") print("="*60) @@ -463,18 +462,18 @@ def main(): parser.add_argument("--iou", type=float, default=0.45, help="IoU threshold for NMS") parser.add_argument("--save", help="Save output video to file") - + args = parser.parse_args() - + # Validate inputs if not Path(args.model).exists(): print(f"[ERROR] Model file not found: {args.model}") return - + if not Path(args.video).exists(): print(f"[ERROR] Video file not found: {args.video}") return - + # Initialize detector detector = YOLOv10Detector( model_path=args.model, @@ -482,7 +481,7 @@ def main(): conf_threshold=args.conf, iou_threshold=args.iou ) - + # Run inference detector.run_inference(args.video, args.save) @@ -522,6 +521,7 @@ docker run -it --rm \ ``` **Expected Console Output:** + ``` [INFO] Initializing OpenVINO Runtime... [INFO] Loading model: /home/openvino/public/yolov10s/FP16/yolov10s.xml @@ -574,6 +574,7 @@ docker run -it --rm \ ### Step 7: Advanced Usage and Features **Save Detection Results:** + ```bash # Save output video with detections docker run -it --rm \ @@ -587,11 +588,13 @@ docker run -it --rm \ ``` **Interactive Controls:** + - **'q'**: Quit application - **'s'**: Save current frame as image - **'p'**: Pause/resume playback **Custom Thresholds:** + ```bash # High precision detection (lower false positives) python3 inference.py --conf 0.6 --iou 0.3 @@ -605,13 +608,15 @@ python3 inference.py --conf 0.3 --iou 0.5 ### YOLOv10s Architecture **Model Characteristics:** + - **Architecture**: Single-stage object detection network -- **Input Size**: 640x640 pixels +- **Input Size**: 640x640 pixels - **Output**: Bounding boxes with confidence scores for 80 COCO classes - **Optimization**: FP16 precision for optimal performance on Intel hardware **Detection Classes:** The model can detect 80 different object classes from the COCO dataset, including: + - **People**: person - **Vehicles**: car, truck, bus, motorcycle, bicycle, airplane, boat, train - **Animals**: bird, cat, dog, horse, sheep, cow, elephant, bear, zebra, giraffe @@ -620,14 +625,16 @@ The model can detect 80 different object classes from the COCO dataset, includin ### Pipeline Architecture **Processing Flow:** + 1. **Input**: Video frame (BGR format from OpenCV) -2. **Preprocessing**: Resize → RGB conversion → Normalization → Tensor formatting +2. **Preprocessing**: Resize → RGB conversion → Normalization → Tensor formatting 3. **Inference**: OpenVINO optimized neural network execution 4. **Postprocessing**: Coordinate conversion → Confidence filtering → Non-Maximum Suppression 5. **Visualization**: Bounding box drawing → Label annotation → Performance overlay 6. **Output**: Annotated frame display **Key Components:** + - **OpenVINO Runtime**: Optimized inference engine for Intel hardware - **YOLOv10s Model**: Pre-trained object detection neural network - **OpenCV**: Computer vision library for image processing and display @@ -636,12 +643,14 @@ The model can detect 80 different object classes from the COCO dataset, includin ### Performance Optimization Features **Hardware Acceleration:** + - **CPU Optimization**: Vectorized operations using Intel® AVX instructions - **GPU Acceleration**: Intel integrated graphics compute units - **Memory Efficiency**: Optimized memory layout and minimal data copies **Software Optimizations:** + - **FP16 Precision**: Reduced memory usage and faster computation -- **Batch Processing**: Single frame inference optimized for real-time performance +- **Batch Processing**: Single frame inference optimized for real-time performance - **Pipeline Parallelism**: Overlapped preprocessing and inference operations - **Efficient NMS**: Optimized Non-Maximum Suppression implementation \ No newline at end of file diff --git a/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-4.md b/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-4.md index 61b148751..25946a850 100644 --- a/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-4.md +++ b/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-4.md @@ -74,6 +74,7 @@ wget -O models/intel/human-pose-estimation-0001/human-pose-estimation-0001.json ### Step 2: Understand Human Pose Estimation Model **Model Architecture:** + - **Model Name**: human-pose-estimation-0001 - **Framework**: OpenVINO optimized model from Intel Model Zoo - **Input**: 256x456 RGB image @@ -114,6 +115,7 @@ docker run -it --rm --net=host \ ``` **Pipeline Execution Features:** + - Real-time human pose detection and visualization - Skeletal overlay on detected persons - Keypoint confidence scoring diff --git a/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-5.md b/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-5.md index a137919c7..de654b4e0 100644 --- a/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-5.md +++ b/metro-ai-suite/metro-sdk-manager/docs/user-guide/metro-vision-ai-sdk/tutorial-5.md @@ -70,7 +70,7 @@ docker run --rm --user=root \ -v "${PWD}:/home/dlstreamer/" \ intel/dlstreamer:2025.1.2-ubuntu24 \ bash -c "export MODELS_PATH=/home/dlstreamer && /opt/intel/dlstreamer/samples/download_public_models.sh yolov10s" - + # Create a continuous DLStreamer pipeline script cat > metro_vision_pipeline.sh << 'EOF' #!/bin/bash @@ -100,7 +100,7 @@ fi # Continuous loop to keep pipeline running while true; do echo "$(date): Starting pipeline iteration..." - + # Run DLStreamer pipeline in Docker container with object detection docker run --rm \ --device /dev/dri:/dev/dri \ @@ -115,7 +115,7 @@ while true; do gvafpscounter ! \ fakesink sync=false \ 2>/dev/null - + echo "$(date): Pipeline ended, restarting in 2 seconds..." sleep 2 done @@ -143,12 +143,14 @@ htop ``` **What to observe:** + - CPU usage per core (bars at the top) - Memory usage and available memory - Running processes sorted by CPU usage - Look for your Metro Vision AI processes **Key shortcuts in htop:** + - `F6` - Sort by different columns (CPU%, MEM%) - `F4` - Filter processes by name - `q` - Quit htop @@ -163,6 +165,7 @@ intel_gpu_top ``` **What to observe:** + - Render/3D engine usage (shows AI inference workload) - Video engine usage (shows video decode/encode) - GPU frequency and power consumption @@ -189,6 +192,7 @@ sudo perf report ``` **What to observe:** + - Function call hotspots and CPU cycles - Call stack and execution paths - Cache misses and memory access patterns @@ -207,7 +211,7 @@ free -h -s 1 | head -20 > memory_usage.log & # Let them run while your AI application is processing ``` -## Step 8: Monitor Process-Specific Performance +## Step 8: Monitor Process-Specific Performance Monitor the specific performance of your AI processes: @@ -231,7 +235,7 @@ Monitor system thermal status during AI workload: # Check CPU temperature sensors | grep Core -# Monitor power consumption (if available) +# Monitor power consumption (if available) sudo powertop --time=10 # Check CPU frequency scaling @@ -265,7 +269,7 @@ Current Memory Usage: $(free -h | awk 'NR==2{printf "Used: %s/%s (%.1f%%)", $3, === Recommendations === - Review htop output for CPU usage patterns -- Check intel_gpu_top for GPU utilization efficiency +- Check intel_gpu_top for GPU utilization efficiency - Analyze perf report for CPU bottlenecks - Monitor thermal status for throttling @@ -285,14 +289,16 @@ echo "kill \$(pgrep -f metro_vision_pipeline)" This tutorial provides a practical approach to profiling Metro Vision AI workloads using command-line tools: ### **What You've Learned:** + 1. **Installing Tools**: Set up `htop`, `intel_gpu_top`, and `perf` for system monitoring -2. **System Monitoring**: Use `htop` for real-time CPU and memory monitoring +2. **System Monitoring**: Use `htop` for real-time CPU and memory monitoring 3. **GPU Profiling**: Monitor Intel GPU performance with `intel_gpu_top` 4. **CPU Profiling**: Use `perf` to identify performance bottlenecks and hotspots 5. **System Analysis**: Collect comprehensive performance data 6. **Performance Reporting**: Generate actionable performance summaries ### **Key Monitoring Points:** + - **CPU Usage**: Monitor core utilization and identify bottlenecks - **Memory Usage**: Track memory consumption and avoid swapping - **GPU Utilization**: Monitor Intel GPU render engine usage diff --git a/metro-ai-suite/metro-sdk-manager/docs/user-guide/support.md b/metro-ai-suite/metro-sdk-manager/docs/user-guide/support.md index 4e4e98ed5..3441f08fd 100644 --- a/metro-ai-suite/metro-sdk-manager/docs/user-guide/support.md +++ b/metro-ai-suite/metro-sdk-manager/docs/user-guide/support.md @@ -3,6 +3,7 @@ This page provides troubleshooting steps, FAQs, and resources to help you resolve common issues. ## Support + - **Developer Forum**: [Join the community forum](#) - **Raise an Issue on GitHub**: [GitHub Issues](https://github.com/open-edge-platform/edge-ai-suites/issues) - **Contact Support**: [Support Page](#) \ No newline at end of file diff --git a/metro-ai-suite/metro-sdk-manager/docs/user-guide/visual-ai-demo-kit/get-started.md b/metro-ai-suite/metro-sdk-manager/docs/user-guide/visual-ai-demo-kit/get-started.md index ae929770a..962882867 100644 --- a/metro-ai-suite/metro-sdk-manager/docs/user-guide/visual-ai-demo-kit/get-started.md +++ b/metro-ai-suite/metro-sdk-manager/docs/user-guide/visual-ai-demo-kit/get-started.md @@ -7,6 +7,7 @@ The Visual AI Demo Kit provides a comprehensive demonstration environment for co ## Learning Objectives Upon completion of this guide, you will be able to: + - Install and configure the Visual AI Demo Kit - Run pre-configured AI applications with real-time dashboards - Execute visual AI inference pipelines on video content @@ -33,6 +34,7 @@ curl https://raw.githubusercontent.com/open-edge-platform/edge-ai-suites/refs/he ![Visual AI Demo Kit Installation](images/visual-ai-demo-kit-install.png) The installation process configures the following components: + - Docker containerization platform - Intel DLStreamer video analytics framework - OpenVINO inference optimization toolkit @@ -82,6 +84,7 @@ docker ps ``` **Expected Services:** + - Grafana Dashboard - DL Streamer Pipeline Server - MQTT Broker @@ -147,19 +150,25 @@ Expand your visual AI expertise with these comprehensive tutorials that demonstr ### Tutorial Series: Advanced Visual AI Applications #### [Tutorial 1: AI Tolling System Tutorial](tutorial-1.md) + Transform the Smart Parking application into a comprehensive AI-based tolling system. This tutorial covers: + - Converting parking detection algorithms to vehicle toll processing - Implementing license plate recognition and vehicle classification - Setting up automated toll calculation and payment processing workflows -#### [Tutorial 2: Customizing Node-RED Flows for Metro Vision AI Applications](tutorial-2.md) +#### [Tutorial 2: Customizing Node-RED Flows for Metro Vision AI Applications](./tutorial-2.md) + Master the art of workflow automation and data processing customization. Learn to: + - Design custom Node-RED flows for visual AI applications - Integrate data sources and external APIs - Build sophisticated data processing pipelines for real-time analytics -#### [Tutorial 3: Customize Grafana Dashboard for Real-Time Object Detection](tutorial-3.md) +#### [Tutorial 3: Customize Grafana Dashboard for Real-Time Object Detection](./tutorial-3.md) + Create compelling visualization experiences for your AI applications. This tutorial demonstrates: + - Building custom Grafana panels and widgets for object detection metrics - Implementing real-time data visualization with dynamic updates - Designing professional dashboards for monitoring and reporting @@ -167,12 +176,31 @@ Create compelling visualization experiences for your AI applications. This tutor ## Additional Resources ### Technical Documentation -- [DLStreamer](http://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/dl-streamer/index.html) - Comprehensive documentation for Intel's GStreamer-based video analytics framework -- [DLStreamer Pipeline Server](https://docs.openedgeplatform.intel.com/edge-ai-libraries/dlstreamer-pipeline-server/main/user-guide/Overview.html) - RESTful microservice architecture documentation for scalable video analytics deployment -- [OpenVINO](https://docs.openvino.ai/2025/get-started.html) - Complete reference for Intel's cross-platform inference optimization toolkit -- [OpenVINO Model Server](https://docs.openvino.ai/2025/model-server/ovms_what_is_openvino_model_server.html) - Model serving infrastructure documentation for production deployments -- [Edge AI Libraries](https://docs.openedgeplatform.intel.com/dev/ai-libraries.html) - Comprehensive development toolkit documentation and API references -- [Edge AI Suites](https://docs.openedgeplatform.intel.com/dev/ai-suite-metro.html) - Complete application suite documentation with implementation examples + +- [DLStreamer](http://docs.openedgeplatform.intel.com/dev/edge-ai-libraries/dl-streamer/index.html) + \- Comprehensive documentation for Intel's GStreamer-based video analytics framework +- [DLStreamer Pipeline Server](https://docs.openedgeplatform.intel.com/edge-ai-libraries/dlstreamer-pipeline-server/main/user-guide/Overview.html) + \- RESTful microservice architecture documentation for scalable video analytics deployment +- [OpenVINO](https://docs.openvino.ai/2025/get-started.html) + \- Complete reference for Intel's cross-platform inference optimization toolkit +- [OpenVINO Model Server](https://docs.openvino.ai/2025/model-server/ovms_what_is_openvino_model_server.html) + \- Model serving infrastructure documentation for production deployments +- [Edge AI Libraries](https://docs.openedgeplatform.intel.com/dev/ai-libraries.html) + \- Comprehensive development toolkit documentation and API references +- [Edge AI Suites](https://docs.openedgeplatform.intel.com/dev/ai-suite-metro.html) + \- Complete application suite documentation with implementation examples ### Support Channels -- [GitHub Issues](https://github.com/open-edge-platform/edge-ai-suites/issues) - Technical issue tracking and community support \ No newline at end of file + +- [GitHub Issues](https://github.com/open-edge-platform/edge-ai-suites/issues) + \- Technical issue tracking and community support + + diff --git a/metro-ai-suite/metro-sdk-manager/docs/user-guide/visual-ai-demo-kit/tutorial-1.md b/metro-ai-suite/metro-sdk-manager/docs/user-guide/visual-ai-demo-kit/tutorial-1.md index 8846a8401..85b44beb8 100644 --- a/metro-ai-suite/metro-sdk-manager/docs/user-guide/visual-ai-demo-kit/tutorial-1.md +++ b/metro-ai-suite/metro-sdk-manager/docs/user-guide/visual-ai-demo-kit/tutorial-1.md @@ -11,6 +11,7 @@ This tutorial walks you through creating an AI-powered tolling system that autom --> By following this guide, you will learn how to: + - **Set up the AI Tolling Application**: Create a new application based on the Smart Parking template and configure it for tolling use cases - **Download and Configure AI Models**: Install YOLO object detection models and Intel's specialized license plate recognition models - **Configure Video Processing Pipeline**: Set up the DLStreamer pipeline for real-time vehicle detection and license plate recognition @@ -31,8 +32,8 @@ By following this guide, you will learn how to: --> ![AI Tolling Sytem Diagram](images/ai-tolling-system.svg) - The AI Tolling system consists of several key components: + - **Video Input**: Processes live camera feeds or video files from toll booth cameras - **Object Detection**: Uses YOLOv10s model to detect vehicles in the video stream - **License Plate Recognition**: Employs Intel's specialized model to extract license plate text @@ -68,6 +69,7 @@ Video File Details The sample video contains: + - Multiple vehicles passing through a toll booth scenario - Various vehicle types (cars, trucks) - Clear license plate visibility for testing recognition accuracy @@ -126,6 +128,7 @@ Model Download Process Details The installation script performs the following operations: + 1. Creates the required directory structure under `src/dlstreamer-pipeline-server/models/` 2. Runs a DLStreamer container to access model download tools 3. Downloads public YOLO models using the built-in download scripts @@ -248,6 +251,7 @@ docker compose up -d ``` The deployment process will: + - Pull required container images - Start the DLStreamer pipeline server - Initialize the Node-RED flow management @@ -265,6 +269,7 @@ docker ps ``` Expected output should show containers for: + - `dlstreamer-pipeline-server` - `node-red` - `grafana` @@ -273,9 +278,10 @@ Expected output should show containers for: ### 2. **Access the Application Interface** Open your web browser and navigate to: + - **Main Dashboard**: `https://localhost/grafana` (Grafana) - - Username: admin - - Password: admin + - Username: admin + - Password: admin - **Node-RED Flow Editor**: `https://localhost/nodered/` ### 3. **Test Video Processing** @@ -322,6 +328,7 @@ For local testing, you can use: `http://localhost/mediamtx/object_detection_1/` ![Vehicle Live Detection](images/car_live_detection.jpg) Expected results: + - Vehicle detection accuracy > 90% - License plate recognition for clearly visible plates - Vehicle attribute classification (car, truck, color) @@ -333,6 +340,7 @@ Expected results: ### 1. **Container Startup Issues** If containers fail to start: + ```bash # Check container logs for specific errors docker logs @@ -346,6 +354,7 @@ docker logs ### 2. **Model Download Failures** If model download fails during installation: + ```bash # Retry the installation with verbose output ./install.sh 2>&1 | tee install.log @@ -360,6 +369,7 @@ df -h ### 3. **Pipeline Processing Errors** If video processing fails or shows poor accuracy: + ```bash # Check pipeline server logs docker logs dlstreamer-pipeline-server @@ -374,6 +384,7 @@ ls -la ./ai-tolling/src/dlstreamer-pipeline-server/models/ ### 4. **Performance Issues** For slow processing or high CPU usage: + - **Reduce video resolution**: Use lower resolution input videos - **Adjust inference device**: Change from CPU to GPU if available - **Optimize pipeline**: Reduce queue sizes or disable unnecessary features diff --git a/metro-ai-suite/metro-sdk-manager/docs/user-guide/visual-ai-demo-kit/tutorial-2.md b/metro-ai-suite/metro-sdk-manager/docs/user-guide/visual-ai-demo-kit/tutorial-2.md index ea69eff69..a9a55c249 100644 --- a/metro-ai-suite/metro-sdk-manager/docs/user-guide/visual-ai-demo-kit/tutorial-2.md +++ b/metro-ai-suite/metro-sdk-manager/docs/user-guide/visual-ai-demo-kit/tutorial-2.md @@ -11,6 +11,7 @@ This tutorial guides you through customizing Node-RED flows to process AI infere --> By following this guide, you will learn how to: + - **Access and Launch Node-RED**: Connect to the Node-RED interface and understand the flow-based programming environment - **Clear and Reset Flows**: Remove existing flows and start with a clean workspace for custom development - **Connect to MQTT Data Streams**: Establish connections to receive real-time AI inference data from metro vision applications @@ -27,8 +28,8 @@ By following this guide, you will learn how to: ## Node-RED Flow Architecture Overview - The custom Node-RED flow consists of: + - **MQTT Input Node**: Subscribes to AI inference data topics - **Function Nodes**: Processes and enhances the incoming data with custom logic - **Debug Nodes**: Provides real-time monitoring of data flow @@ -58,10 +59,13 @@ Troubleshooting Node-RED Access If you cannot access Node-RED: + 1. Verify the metro vision AI application is running: + ```bash docker ps | grep node-red ``` + 2. Check that port 1880 is exposed and accessible 3. Ensure no firewall is blocking the connection 4. Try accessing via localhost if running on the same machine: `http://localhost:1880` @@ -98,7 +102,6 @@ Set up an MQTT subscriber node to receive AI inference data: - **Name**: `AI Inference Input` - Click **Done** to save the configuration - ### 4. **Add Debug Output for Monitoring** Create a debug node to monitor incoming data: @@ -146,7 +149,6 @@ Create a debug node to monitor incoming data: After running this command, you should see AI inference data appearing in the Node-RED debug panel. - ### 5. **Implement Custom Data Processing Function** Add a function node to enhance the AI inference data with custom metadata: @@ -176,7 +178,7 @@ for (let obj of msg.payload.objects) { if (!obj.license_plate || !obj.color || !obj.type) { continue; // Skip this object if missing any attribute } - + // Extract the data let extractedObj = { license: obj.license_plate.label || null, @@ -186,7 +188,7 @@ for (let obj of msg.payload.objects) { color_confidence: obj.color.confidence || null, type_confidence: obj.type.confidence || null }; - + extractedData.push(extractedObj); } @@ -200,7 +202,6 @@ msg.payload = extractedData; return msg; ``` - ### 6. **Configure MQTT Output for Enhanced Data** Set up an MQTT publisher to send the enhanced data: @@ -232,7 +233,6 @@ Test your custom Node-RED flow: - Verify that both raw and enhanced data are flowing through the system - Check timestamps and custom metadata are being added correctly - ## Expected Results ![Node Red Flow](images/node-red-flow.png) @@ -253,8 +253,10 @@ After successfully setting up the AI Tolling system with Node Red, consider thes ## Troubleshooting ### **Node-RED Interface Not Accessible** + - **Problem**: Cannot access Node-RED at the specified URL -- **Solution**: +- **Solution**: + ```bash # Check if Node-RED container is running docker ps | grep node-red @@ -263,15 +265,17 @@ After successfully setting up the AI Tolling system with Node Red, consider thes ``` ### **No Data in Debug Panel** + - **Problem**: Debug nodes show no incoming data -- **Solution**: +- **Solution**: - Verify the AI application is running and generating inference data - Check MQTT topic names match your application's output topics - Ensure proper JSON parsing in function nodes ### **Function Node Errors** + - **Problem**: Function node shows errors in the debug panel -- **Solution**: +- **Solution**: - Add try-catch blocks around JSON parsing - Use `node.warn()` or `node.error()` for debugging - Validate input data structure before processing diff --git a/metro-ai-suite/metro-sdk-manager/docs/user-guide/visual-ai-demo-kit/tutorial-3.md b/metro-ai-suite/metro-sdk-manager/docs/user-guide/visual-ai-demo-kit/tutorial-3.md index cb29a2b6a..51d0d6eea 100644 --- a/metro-ai-suite/metro-sdk-manager/docs/user-guide/visual-ai-demo-kit/tutorial-3.md +++ b/metro-ai-suite/metro-sdk-manager/docs/user-guide/visual-ai-demo-kit/tutorial-3.md @@ -9,6 +9,7 @@ This tutorial guides you through creating a custom Grafana dashboard that displa **What You Can Do**: Highlight the developer workflows supported by the guide. --> By following this guide, you will learn how to: + - **Create Custom Dashboards**: Build new Grafana dashboards tailored for metro vision applications - **Integrate Video Streams**: Embed real-time WebRTC video feeds using HTML panels - **Visualize MQTT Data**: Create dynamic tables that display live object detection data from MQTT streams @@ -46,9 +47,9 @@ By following this guide, you will learn how to: - In the below code update to your host IP address. If you are testing on localhost, update it to localhost. ```html - @@ -59,7 +60,6 @@ By following this guide, you will learn how to: - Click "Save Dashboard" to save the panel - Adjust panel size as needed. - ### 3. **Create MQTT Data Table** 1. **Add New Panel**: @@ -73,8 +73,7 @@ By following this guide, you will learn how to: - Update Topic to "inference/enhanced" 3. **Add Transformations** (Optional): - - You can add different types of transformations to this dashboard panel. - + - You can add different types of transformations to this dashboard panel. ### 4. **Configure Dashboard Layout**