Gavin Whitlock

How I built the assistant

The assistant on my homepage is a retrieval-augmented generation (RAG) app. It searches a small set of files about my work, then asks an open-weight model to answer using only what it found. Here is each step and why I built it that way.

  1. Step 1

    Browser

    Sends your question, recent messages and persona to /api/chat.

    React client

  2. Step 2

    API route

    Validates input, rate-limits per visitor, keeps the last 8 turns.

    Next.js on Vercel

  3. Step 3

    Retriever

    BM25 ranks sections of my resume and notes against the question.

    LangChain BaseRetriever

  4. Step 4

    Prompt

    Rules, persona, a fixed profile, retrieved sections and history.

    ChatPromptTemplate

  5. Step 5

    Model

    Writes the answer from that context only.

    Kimi K3 via Moonshot API

  6. Step 6

    Stream

    Tokens stream back; source sections ride in a response header.

    ReadableStream

Knowledge base

Markdown files in /content plus structured resume data in lib/profile.ts, split into one chunk per section at startup.

Upstash Redis

Stores chat logs and contact details, and backs the rate limiter. Optional: without it, logs go to the console.

The job fit check follows the same path, but asks the model for structured JSON instead of streamed text.

Why Kimi K3, an open-weight model

Most assistants like this call a closed model. I wanted to learn what changes with an open-weight one. Moonshot AI publishes Kimi K3's weights on Hugging Face, so the same model could run on my own GPUs with vLLM or SGLang, or on Amazon Bedrock, without rewriting the app.

In practice K3 is a 2.8-trillion-parameter mixture-of-experts model, far too big for Vercel, so I call Moonshot's hosted API. That API is OpenAI-compatible, which means LangChain's standard ChatOpenAI class works once it points at a different URL:

const model = new ChatOpenAI({
  model: "kimi-k3",
  apiKey: process.env.MOONSHOT_API_KEY,
  configuration: { baseURL: "https://api.moonshot.ai/v1" },
  modelKwargs: { reasoning_effort: "low" },
});

Two K3 details mattered. It always runs in thinking mode, and the default effort is max, which is too slow for chat, so I set it to low. And K3 fixes its own sampling settings, so the request must not include temperature or top_p. My test suite runs the chain against a fake server and checks for both.

Why LangChain.js

I write JavaScript and Node day to day, and Next.js runs natively on Vercel, so LangChain.js kept everything in one language and one repo. I use four of its building blocks: Document and a text splitter for the knowledge base, a BaseRetriever for search, ChatPromptTemplate for the prompt, and withStructuredOutput for the fit check. Each piece can be replaced on its own.

const docs  = await retriever.invoke(question);
const chain = chatPrompt.pipe(kimi()).pipe(new StringOutputParser());
const stream = await chain.stream({ persona, profile, context: format(docs), history, question });

Retrieval without a vector database

RAG tutorials usually start with an embeddings model and a vector store. Moonshot doesn't offer an embeddings endpoint, and my knowledge base is only a few thousand words, so I wrote a 60-line BM25 keyword retriever. It runs in memory, adds no service, and is easy to debug because you can see exactly why a section matched.

Each ## section of a markdown file becomes one chunk, so a retrieved chunk is always a complete thought with its heading. A short profile goes into every prompt no matter what, so basic questions never depend on retrieval. If the knowledge base grows, swapping in embeddings is one line, because anything that extends BaseRetriever fits the same slot.

Structured output for the fit check

When a hiring manager pastes a job description, a free-text answer is hard to scan. The fit route defines a Zod schema with a summary, matches with evidence, and gaps, and LangChain converts it to a JSON schema that K3 is required to follow. The prompt asks for honest gaps, because a fit check that always says yes isn't useful.

const model = kimi().withStructuredOutput(FitSchema, {
  name: "job_fit", method: "jsonSchema", strict: true,
});

What I left out, and why

There's no agent loop yet. With tool calling, K3 requires the full assistant message, including its hidden reasoning, to be sent back unchanged on the next turn. That's easy to get wrong through an adapter, and a chain answers faster with fewer ways to fail. Contact capture uses a plain form instead of a tool call.

Next I plan to move to a LangGraph agent with tools for lead capture and booking time, add an evaluation set of real questions from the chat logs, and compare K3 with other open-weight models through the same interface.

Guardrails and privacy

The system prompt limits answers to the knowledge files and tells the model to say when it doesn't know rather than guess. Inputs are length-capped, only recent turns are sent, and each visitor is rate-limited. API keys stay on the server. Visitors are told under the chat box that conversations are saved.