What building an AI documentation assistant taught me about technical writing

Most technical writers don't build AI agents. We document them. We sit in on the sprint review. We read the architecture decision records. Then we figure out how to explain a retrieval pipeline to someone who needs their integration working by Friday.

I built one anyway.

His name is Artie, an AI documentation assistant I built with Algolia. He runs inside this documentation site. Building him taught me more about writing and AI than any conference talk could.

This isn't a walkthrough of Artie's design. The case study covers that. This post covers the broader lessons:

• What AI agents actually are
• How retrieval-augmented generation works in practice
• Why prompt engineering is closer to technical writing than most people realize
• What this all means for people in this little thing of ours

What's an AI agent and what isn't

People apply the term "AI agent" to everything from spell-checkers to autocomplete. The differences matter. Not every AI feature is an agent. A spell-checker that uses machine learning isn't one. An autocomplete suggestion isn't one. A chatbot that follows a fixed decision tree is a flowchart with a text box.

An AI agent is a system that can interpret a goal and act on it within a defined environment. Autonomy within limits separates an agent from a prompt-response loop. The agent reads context and picks a strategy, sometimes over more than one step.

Where does Artie sit on this spectrum?

He's not an agent by that definition. He doesn't browse the web, run code, or chain together multi-step tool calls. Algolia retrieves the pages before the model runs. The model receives them as context. What he does do is interpret intent and apply guardrails on his own. He works out whether a question is technical or a portfolio inquiry before he answers. That's more than a search box with a text field. It's less than a fully autonomous agent that could, say, file a bug report from a user's question.

Technical writers need this distinction now. Understanding AI tools and agents is part of the job. When you document an AI-powered product, you need to know where it falls on the spectrum. Then you need to tell users what "AI" means in that product, because their expectations vary.

How retrieval-augmented generation actually works

Retrieval-augmented generation (RAG) powers Artie and a growing number of AI documentation tools. RAG works around a core limit of large language models (LLMs). A model trains on a fixed dataset. Your content is absent from its training data.

A standard LLM interaction works like this: the user sends a prompt, and the model generates a response from its training data. That works for general knowledge and fails for domain-specific questions. Ask a base model about your product's API and it either invents an answer or declines.

RAG inserts a retrieval step between the user's question and the model's response. The pipeline looks like this:

1. Query processing. The retrieval layer converts the user's question into a vector embedding, a numeric representation of its meaning.
2. Retrieval. The system searches the knowledge base for content with similar meaning. Here, the knowledge base is a pre-built index of your documentation. Ranking depends on meaning, often alongside keyword matches. Queries like "how do I set up authentication?" and "configure login credentials" return the same documentation page.
3. Context assembly. The system packages the retrieved documents, or chunks of them, into a context window next to the user's question and the system prompt. The model then processes that package.
4. Generation. The model generates a response grounded in the retrieved context rather than its training data. In a well-tuned system, the model cites its sources and stays within what the retrieved documents say.

Mermaid (image)

Mermaid (code)

flowchart TD
    A["🔍 User Question"] --> B["Query Processing"]
    B -->|Vector embedding| C["Retrieval"]
    KB[("📚 Knowledge Base")] -.-> C
    C -->|Relevant docs| D["Context Assembly"]
    D -->|Prompt + context| E["Generation"]
    E --> F["📄 Grounded Response"]

Quality depends on step 2. If the retrieval layer returns the wrong documents, or ranks them badly, model capability doesn't help. The model writes a plausible, well-structured answer from the wrong source material. Chunking and embedding-model choice matter more than most teams expect. The generation model gets the credit, but the retrieval layer determines quality.

In Artie's case, the knowledge base is this site's documentation. Algolia indexes and retrieves it. Artie covers only what's documented here, which keeps him inside the site's own material. The case study covers how that plays out. RAG doesn't make a model smarter. It makes a model more accountable, because every response ties back to a verifiable source.

Prompt engineering is technical writing

The project taught me one big lesson. A system prompt is a technical writing deliverable.

A well-crafted system prompt contains the same parts as a style guide. You define the audience. You establish tone and voice rules. You set scope boundaries: what the system should address and what it shouldn't. You write instructions precise enough for a machine to follow without human judgment. You order the information so the most critical rules take precedence.

The work is audience analysis and a style guide. It's also a content plan.

The skills transfer is almost one-to-one:

• Audience analysis → Audience routing. Technical writers identify the audience and tailor content to it, and a system prompt does the same for an AI. Artie's prompt routes between technical users and recruiters, the way a well-structured documentation site routes between reader personas.
• Style guide → Persona calibration. Every documentation team has voice and tone guidelines. A system prompt is a style guide for an AI. It sets tone per audience and requires a source link for every claim.
• Information architecture → Instruction hierarchy. Technical writers know that order and structure affect comprehension. Instruction placement changes how well a model follows a rule.
• Edge case documentation → Guardrail design. Good documentation covers the unexpected action with clear guidance. Good prompt design covers the user who tries to break the system.

Not every technical writer needs to become a prompt engineer. The discipline borrows from skills technical writers build over decades. If you can write a clear, structured, audience-aware document, you bring more to this work than you think.

Guardrails are a language problem

People treat AI security as a tech problem: rate limits, logins, access controls. Those matter. In a public chatbot, much of what attackers target is linguistic.

info

Prompt injection means a user overrides the system prompt. A common example is the line "ignore all previous instructions."

Prompt injection uses words to get around word-based controls. Defense means writing rules clear enough that an attacker can't talk the model past them.

The Open Worldwide Application Security Project (OWASP) list of the top 10 risks for LLM apps names prompt injection the top risk. Read through that list and you'll notice how many risks are, at root, language problems:

• Improper output handling (what the model outputs)
• Data and model poisoning (what the model learned from text)
• Excessive agency (what the prompt tells the model it can do)

The exploits are technical. The defenses are often plain language, written into the system prompt.

That reframes security work for technical writers. Nobody expects us to build authentication systems. But we can write the rules that define how an AI responds. We can write them precisely enough to survive adversarial input.

What changes for documentation work

I won't predict the future of AI. I can describe the trajectory visible right now and what it means for people who document software for a living.

AI documentation assistants moved from novelty to expectation. Users no longer accept static search. They want to ask a plain question and get a grounded answer instead of ten blue links. RAG systems make that possible now. The tooling keeps getting easier to adopt.

tip

Want the infrastructure side? The Event Streams & Observability Pipelines docs show how Datadog and Galileo track model calls.

The role of the technical writer expands rather than shrinks. Somebody has to curate the knowledge base these systems retrieve from. Somebody has to write the system prompts that define how they respond and where they stop. Somebody has to test whether the responses match the documentation. That's all writing work, and it demands judgment about how people consume information.

Technical writers should learn how LLMs and RAG systems work. We shouldn't become machine learning engineers, but we should join conversations about AI products. We need enough pipeline knowledge to know where our expertise applies and where it doesn't.

Technical writers already write for many audiences. They structure content and set scope. They plan for misuse. Those skills apply directly to AI work.

AI won't replace technical writers. The models get better at generating text each year, but someone still decides what to say, and to whom, within stated limits.

They need an editor. We're the editors.

Writing the first draft yourself works too. Point the AI agent at it as a proofreader and style checker.

How you use AI in your workflow is your call, like every other tool at our disposal.

The job isn't changing. The tools are changing. The work isn't going anywhere. Teams that cut their technical writers come back for them.

Read the case study

If this post gave you the "why it matters" and "how it works," read the case study. It covers the specific design decisions behind Artie:

• Audience routing
• Knowledge grounding
• Persona calibration
• Security guardrails
• Tradeoffs I'd reconsider for a production deployment

Each piece works on its own. Start with either one.