How I built the assistant
The assistant on my homepage is a retrieval-augmented generation (RAG) app. It searches a small set of files about my work, then asks an open-weight model to answer using only what it found. Here is each step and why I built it that way.
Step 1
Browser
Sends your question, recent messages and persona to /api/chat.
React client
Step 2
API route
Validates input, rate-limits per visitor, keeps the last 8 turns.
Next.js on Vercel
Step 3
Retriever
BM25 ranks sections of my resume and notes against the question.
LangChain BaseRetriever
Step 4
Prompt
Rules, persona, a fixed profile, retrieved sections and history.
ChatPromptTemplate
Step 5
Model
Writes the answer from that context only.
Kimi K3 via Moonshot API
Step 6
Stream
Tokens stream back; source sections ride in a response header.
ReadableStream
Knowledge base
Markdown files in /content plus structured resume data in lib/profile.ts, split into one chunk per section at startup.
Upstash Redis
Stores chat logs and contact details, and backs the rate limiter. Optional: without it, logs go to the console.
Why Kimi K3, an open-weight model
Most assistants like this call a closed model. I wanted to learn what changes with an open-weight one. Moonshot AI publishes Kimi K3's weights on Hugging Face, so the same model could run on my own GPUs with vLLM or SGLang, or on Amazon Bedrock, without rewriting the app.
In practice K3 is a 2.8-trillion-parameter mixture-of-experts model, far too big for Vercel, so I call Moonshot's hosted API. That API is OpenAI-compatible, which means LangChain's standard ChatOpenAI class works once it points at a different URL:
const model = new ChatOpenAI({
model: "kimi-k3",
apiKey: process.env.MOONSHOT_API_KEY,
configuration: { baseURL: "https://api.moonshot.ai/v1" },
modelKwargs: { reasoning_effort: "low" },
});Two K3 details mattered. It always runs in thinking mode, and the default effort is max, which is too slow for chat, so I set it to low. And K3 fixes its own sampling settings, so the request must not include temperature or top_p. My test suite runs the chain against a fake server and checks for both.
Why LangChain.js
I write JavaScript and Node day to day, and Next.js runs natively on Vercel, so LangChain.js kept everything in one language and one repo. I use four of its building blocks: Document and a text splitter for the knowledge base, a BaseRetriever for search, ChatPromptTemplate for the prompt, and withStructuredOutput for the fit check. Each piece can be replaced on its own.
const docs = await retriever.invoke(question);
const chain = chatPrompt.pipe(kimi()).pipe(new StringOutputParser());
const stream = await chain.stream({ persona, profile, context: format(docs), history, question });Retrieval without a vector database
RAG tutorials usually start with an embeddings model and a vector store. Moonshot doesn't offer an embeddings endpoint, and my knowledge base is only a few thousand words, so I wrote a 60-line BM25 keyword retriever. It runs in memory, adds no service, and is easy to debug because you can see exactly why a section matched.
Each ## section of a markdown file becomes one chunk, so a retrieved chunk is always a complete thought with its heading. A short profile goes into every prompt no matter what, so basic questions never depend on retrieval. If the knowledge base grows, swapping in embeddings is one line, because anything that extends BaseRetriever fits the same slot.
Structured output for the fit check
When a hiring manager pastes a job description, a free-text answer is hard to scan. The fit route defines a Zod schema with a summary, matches with evidence, and gaps, and LangChain converts it to a JSON schema that K3 is required to follow. The prompt asks for honest gaps, because a fit check that always says yes isn't useful.
const model = kimi().withStructuredOutput(FitSchema, {
name: "job_fit", method: "jsonSchema", strict: true,
});What I left out, and why
There's no agent loop yet. With tool calling, K3 requires the full assistant message, including its hidden reasoning, to be sent back unchanged on the next turn. That's easy to get wrong through an adapter, and a chain answers faster with fewer ways to fail. Contact capture uses a plain form instead of a tool call.
Next I plan to move to a LangGraph agent with tools for lead capture and booking time, add an evaluation set of real questions from the chat logs, and compare K3 with other open-weight models through the same interface.
Guardrails and privacy
The system prompt limits answers to the knowledge files and tells the model to say when it doesn't know rather than guess. Inputs are length-capped, only recent turns are sent, and each visitor is rate-limited. API keys stay on the server. Visitors are told under the chat box that conversations are saved.