Turning 1,000+ transcripts into an AI-searchable knowledge system.
A large private corpus of teachings was valuable — and nearly unusable at scale. I designed and built Dharma Sky: a private library where those transcripts can be browsed, searched, and asked about through an AI interface grounded in the source material.
Role: Product Designer · AI Product Builder
AI Product Design
AI-assisted Development
Knowledge Retrieval
Product Building
The challenge
Knowledge trapped inside a thousand files
The material existed — years of transcripts across formats and folders. Finding a teaching on a theme, comparing related passages, or answering a precise question meant manual hunting through an archive no human interface could reasonably navigate.
This was an information architecture problem: how to make a large, unstructured body of knowledge accessible without asking someone to remember filenames, dates, or which document held which idea.
The idea
A retrieval layer between people and the archive
Instead of asking users to search documents one by one, Dharma Sky processes the corpus into an indexed knowledge layer — then offers a calm product surface for browsing, thematic exploration, and grounded questions.
The system
From unstructured transcripts to grounded answers
The core product story is the indexing pipeline. Everything the user experiences — search, Ask, topics — depends on what happens here.
Indexing pipeline
01
Transcripts
A private corpus of 1,000+ teachings arrives in many formats — Word, PDF, RTF, HTML, scanned images, nested archives.
02
Processing
Each file is expanded, text is extracted (including OCR for images), and the transcript is normalized into readable teaching text.
03
Indexing
The teaching is split into retrieval chunks, embedded for vector search, and also given a document-level index: key terms, a faithful synopsis, and a routing embedding.
04
Retrieval
A question first shortlists the most relevant teachings, then searches passages inside those teachings — so answers stay grounded in the right sources.
05
AI response
The model answers only from retrieved passages, with citations back to the original teachings — never inventing content beyond the library.
Chunking
Teachings are split into overlapping retrieval units sized for embedding — keeping page context when available.
Embeddings
Chunks and document indexes are embedded so meaning — not just keywords — can be searched.
Hierarchical retrieval
Shortlist teachings first, then retrieve passages inside them. Diversify sources so one file doesn’t dominate the answer.
The point for a recruiter who has never built AI infrastructure: I understand what happens between a user asking a question and an AI producing an answer — and I designed that path as a product.
Architecture evolution
A hybrid system built for speed, cost, and control
As the corpus grew, I moved retrieval away from a single Supabase dependency. Each service now does the job it is best at, while the product keeps one consistent experience.
01
System of record
Supabase
Authentication, private file storage, structured metadata, topics, and user data remain in Supabase.
02
Retrieval engine
Qdrant
Document and chunk embeddings live in Qdrant. Passage text and source metadata travel with each vector result, removing an extra database round trip from Ask.
03
Experience layer
Next.js cache
Catalog, teaching, and topic reads use tagged server caches. Storage changes trigger signed revalidation, so repeated browsing stays efficient without serving stale content.
Staged migration
A shared retrieval interface can route to Qdrant or the existing Postgres vector functions by configuration. This separated product behavior from storage choice and made the migration incremental.
UX effect
People keep the same simple model — browse, ask, verify sources — while caching and payload-rich vector results reduce avoidable backend work.
Scale
1,000+
transcripts
At this scale, manually searching the source material is no longer a practical interface.
The product needed a structured way to process, index, retrieve, and surface the right knowledge — so a question or theme could open the archive without requiring someone to know where each teaching lived.
Product & UX
A simple surface over a complex system
The interface is intentionally quiet: library, browse, topics, Ask. Complexity stays in the pipeline; the product asks only for intent.
Ask
Natural-language questions become a two-stage search: find the right teachings, then quote from matching passages.
Browse
Semantic search ranked by aboutness — synopses and key terms make a large library scannable without opening every file.
Library
Catalog filters by year, event, type, and topic. Detail pages open the full transcript with optional AI summaries.
Topics
Themes gathered across teachings, with Topic Builder linking each theme back to supporting passages.
How people move through it
Search by meaning — not only exact titles — using the document index and semantic ranking.
Open a teaching to read the transcript, scan source metadata, and optionally generate a structured summary.
Explore curated topics that pull passages from across the corpus instead of forcing a single-file mental model.
Ask in natural language when the goal is an answer with sources — not a file hunt.
Browse
Anonymized preview
How to work with impatience
Teaching on patience · Event archive
Synopsis highlights working with irritation, softening reaction, and returning to practice.
Evening talk · Mind training
Key terms surface themes of anger, patience, and daily application — without opening the full file first.
AI interaction
Conversational — but purpose-built
Ask Dharma is not a generic chat clone. It is designed around one job: answer from this library, show the passages that support the answer, and refuse to invent teachings that are not there.
How should we practice when impatience arises?
From the library
Across the retrieved teachings, impatience is treated as something to recognize early — then meet with patience practice rather than feeding the reaction. The passages emphasize returning to a calm mind and applying the instruction in ordinary moments.
Supporting passages
“When impatience appears, do not follow it — recognize it, and apply the method you have learned.”
Source teaching · anonymized title
Illustrative exchange — private corpus content is not shown.
Under the surface: the question is embedded, candidate teachings are shortlisted from the document index (and lexical key-term overlap), matching chunks are retrieved, sources are diversified, and only then does the model write — constrained to those passages.
The model does not "know" the transcripts. It answers from what retrieval returns.
Building the product
Designed it. Built it. Shipped it.
I used AI-assisted tools to move from product concept and UX into a working application — schema, ingestion pipeline, retrieval, and interface.
The important part is not that AI wrote code. It is that I could direct the full loop: define the problem, design the experience, decide how knowledge should be indexed and retrieved, and ship a private product people can actually use.
Multi-format ingestion pipeline
Supabase + Qdrant hybrid architecture
Document-level + chunk-level indexes
Grounded Ask flow with citations
Tagged cache + event-driven invalidation
Authenticated private library shell
Design decisions
Product choices that shaped the system
Hide the infrastructure
Users never see embeddings, chunk boundaries, or vector scores. They see search, answers, and sources.
Retrieve before you answer
The model is not treated as if it "knows" the corpus. Passages are retrieved first; the answer is constrained to those passages.
Cite every claim path
Supporting passages appear under each answer, linked to teaching titles — trust comes from provenance, not polish.
Index at two levels
Document-level aboutness finds the right teachings; chunk-level search finds the exact lines to quote.
Design for private knowledge
Authenticated access, private storage, and no public registration — the corpus stays a closed library.
States that match the system
Loading (“Searching the teachings…”), empty results, and retrieval failures are designed as product moments, not generic errors.
Split the system by responsibility
Supabase manages private content and relational data; Qdrant handles semantic retrieval and returns passage payloads with each result.
Cache without losing freshness
Stable library reads are cached by tag, then automatically revalidated when teaching files change.
What I learned
Building AI products is systems design
1
Designing an AI product means designing the system around the AI — retrieval, grounding, and citations — not dropping a chatbot onto a document pile.
2
Retrieval quality is product quality. If the wrong passages surface, the answer feels broken even when the UI is calm.
3
Good AI UX makes complex infrastructure invisible. Users should feel guided, not impressed by jargon.
4
Architecture choices should map to user value: fewer backend round trips and deliberate cache invalidation support a faster-feeling, dependable library.
5
AI-assisted development changed how quickly I could move from product concept and UX into a working system I could iterate on.
Product Designer · AI Product Builder
Problem → product thinking → UX → AI system → indexing → shipped software
