Ostad AI is one product I shipped across two messaging channels — a WhatsApp tutor in NestJS and a Telegram tutor in Bun + grammY, each its own implementation unified under one umbrella rather than a single shared codebase. Both answer typed questions, photos of textbook problems, and Bangla voice notes, and both reply in Bangla with a properly typeset PDF (KaTeX math, Noto Bengali type) instead of raw text.
Everything is grounded in a knowledge base I built myself: image-based NCTB/HSC Bengali-medium textbooks run through Gemini-vision OCR, block-aware chunking, and embeddings into Supabase pgvector, which both channels query by RPC. The KB went through three generations — an OCR spike, a first end-to-end ICT pipeline with dual-embedding retrieval, and a PRD-driven production rewrite with full ingestion provenance and HNSW indexing.
The hardest problem was cross-lingual retrieval for a low-resource language: a validation set exposed Bengali questions returning zero similarity, so I designed a dual-embedding scheme (content + a Gemini-generated “question-form” vector) plus a bilingual concept vocabulary that script-splits OCR tags and back-fills the missing language — so a Banglish student question and English textbook prose land on the same chunk.
The hard part
Cross-lingual retrieval for a low-resource language
A validation set exposed the core failure: several Bengali questions returned zero similarity against English-medium textbook chunks. I designed a dual-embedding scheme — every chunk gets a content vector and a Gemini-generated “what question would this answer?” vector — paired with a bilingual concept vocabulary that splits OCR tags by Unicode script (Bengali block vs Latin) and back-fills the missing language from a per-subject term dictionary. The result: a Banglish or Bengali student question and the English textbook prose that answers it land on the same chunk.
Highlights
- Built the entire NCTB-textbook RAG knowledge base across three repo generations — a Gemini-vision OCR spike, a first end-to-end ICT pipeline (OpenAI text-embedding-3-small, 1536-dim, dual-embedding retrieval), and a PRD-driven production rewrite (Gemini embeddings 3072-dim, HNSW, normalized documents/pages/chunks/ingestion-runs provenance).
- Solved cross-lingual retrieval for Bengali: a dual-embedding scheme (content + a Gemini-generated “question-form” vector) plus a bilingual concept_vocab that script-splits OCR tags by Unicode block and back-fills the missing language, targeting the zero-similarity Bengali misses from the 18/32 validation run.
- Built a 7-node LangGraph pipeline on the WhatsApp tutor: supervisor classifier → RAG retrieval → complexity-tiered specialists (2/3/4 agents in parallel via Promise.all) → LLM-judge validation → pure-function merger, with a Groq → Gemini → OpenAI failover chain that degrades instead of crashing.
- Built the Telegram tutor as a thin agentic Gemini router over 5 function-calling tools (answer, explain, generate questions, grade, voice), pushing routing into rich tool descriptions instead of brittle keyword matching.
- Engineered production-grade ingestion safety: content-addressed (SHA-256) documents, is_active chunk supersession, page-scoped deactivation so retrying failed OCR pages never wipes good chunks, and a per-page confidence heuristic that flags low-quality pages for opt-in larger-model re-OCR.
Stack