Models & frameworks
Video + pose estimation
MediaPipe Pose Landmarker (BlazePose, lite, TFLite)33 body keypoints in image and metric 3D (world) coordinates per frame; OpenCV for decoding and frame handling.
Keypoint cleaning + features
NumPy · pandasLow-confidence flagging, gap interpolation, moving-average smoothing; torso-length normalisation; wrist-speed phase detection; joint angles.
Stroke check
Rule-based pose-sequence classifierHitting-wrist path and reach decide forehand / backhand / serve and return a confidence score.
Expert benchmark
THETIS expert forehands (53 clips)Feature distribution of expert players is the reference; each measurement is expressed as an expert percentile.
Retrieval (RAG)
all-MiniLM-L6-v2 bi-encoder + FAISS (IndexFlatIP)Dense retrieval over passage-level chunks of the coaching documents: top-8 candidates by cosine similarity.
Reranking
cross-encoder/ms-marco-MiniLM-L-6-v2Cross-encoder rescoring of the 8 candidates; the top-3 passages become the evidence for the answer.
Answer generation
Qwen2.5-3B-Instruct (Hugging Face transformers, bf16)Grounded generation: question + measurements + retrieved passages → answer, reasoning, improvement steps, with [n] citations.
Practice planning
Rule-based adaptive plannerDrill library × time × court access × partner; adjusts levels from completion, difficulty and new video observations.
Application
Next.js 15 (App Router, TypeScript) + Python 3.13API routes serve results and run the Python pipeline for new uploads.