SignMaster
AI-Powered Filipino Sign Language Platform
The Challenge
Filipino Sign Language (FSL) has distinct regional grammatical structures that differ from American Sign Language (ASL). No existing open-source translation models accommodated native FSL vocabulary for elementary deaf students.
How It Was Solved
Curated and trained a custom FSL gesture dataset. Deployed an on-device computer vision pipeline leveraging Google MediaPipe landmark detection coupled with a lightweight TensorFlow deep learning classification model.
System Architecture & Data Flow
Real-Time Computer Vision Inference Pipeline
High-throughput frame processing extracting 3D hand landmarks for low-latency neural classification.
Video Ingestion
Webcam video frame capture at 30+ FPS normalized for diverse ambient lighting conditions.
Landmark Extraction
21 three-dimensional coordinate keypoints extracted per hand, eliminating redundant background pixels.
Coordinate Normalization
Translation and scale-invariant geometric normalization relative to wrist anchors.
Neural Inference
Trained deep neural network classifying spatial hand conformations into FSL vocabulary.
User Feedback Interface
Audio-visual speech synthesis and real-time textual feedback for classroom learners.
Key Features & Engineering Highlights
Native FSL Landmark Model
Custom-curated dataset recorded and validated in conjunction with accredited FSL instructors.
Low-Latency Edge Inference
Optimized keypoint tensor inputs allowing smooth 30+ FPS real-time prediction even on consumer laptops without discrete GPUs.
Architectural Trade-offs & Decisions
Decision 1: MediaPipe Coordinate Extraction over Raw CNN Image Frames
Feeding 21 normalized coordinates to a lightweight model is 100x more computationally efficient and privacy-preserving than processing full RGB camera frames.
Requires adequate lighting for initial landmark triangulation.
Measurable Outcomes & Production Impact
- ✓Achieved a 4.75 out of 5 user evaluation score across accuracy and classroom usability metrics.
- ✓Demonstrated in real-world educational evaluations with SNED students and instructors.