Every company has the same quiet productivity drain: people hunting for the latest version of a policy, asking the one colleague who knows how a process works, or rewriting answers that already exist somewhere in a shared drive. An internal AI assistant that can search and summarize your own documents addresses exactly that. This is a practical build plan, written for the person who has to get it approved, scoped and adopted.

Step 1: Pick one audience and one job

The most common mistake is trying to build “ChatGPT for the whole company” on day one. Start with a single team and a clear set of questions. Good first candidates include:

  • New-hire onboarding: tools, processes, who to ask for what.
  • Customer support agents looking up product details and policies.
  • Sales teams searching proposals, case notes and technical specs.
  • Operations staff following standard operating procedures.

Write down twenty to fifty real questions this group asks. That list becomes your scope, your test set and your success criteria.

Step 2: Audit the content

Your assistant can only be as good as what it reads. For the chosen team, map where the answers live and how trustworthy they are.

SourceTypical issueWhat to do
Shared drivesMany outdated copiesArchive old versions, keep one source of truth
Wiki or intranetPages without ownersAssign an owner per section
PDFs and slide decksText locked in layouts and imagesCheck text extraction quality, convert key files
Ticket or chat historyMixed quality, personal dataUse selectively, filter sensitive fields
SpreadsheetsStructure lost when treated as textQuery them as data instead of as documents

Step 3: Design permissions first

This is the step most prototypes skip and most production systems regret skipping. If a document is restricted in your file system, the assistant must not reveal it to someone without access. That means filtering at retrieval time based on the user’s identity, not hiding things in the interface.

Practical rules:

  • Sign users in with your existing identity provider rather than separate accounts.
  • Store access information alongside each indexed chunk of content.
  • Log who asked what and which documents were used for each answer.
  • Exclude highly sensitive categories, such as salaries or medical information, unless there is a specific, approved need.

Step 4: Choose the architecture

Under the hood, most internal assistants use retrieval-augmented generation: search your content, pass the most relevant passages to a language model and ask it to answer with citations. The main decision is how much to build versus buy.

  • Built-in assistants from your workspace suite are quick to switch on, but you have limited control over sources, behavior and data handling.
  • Configurable platforms let you connect sources and adjust behavior, with some limits on custom workflows.
  • A custom build gives full control over permissions, interface, integrations and model choice, at the cost of development and maintenance.

Our article on custom software vs off-the-shelf walks through that trade-off in more general terms.

Step 5: Build the minimum useful version

A first version needs fewer features than people expect:

  1. A connector that syncs content from one or two sources on a schedule.
  2. Chunking and indexing with metadata such as title, owner, date and access group.
  3. A chat interface where people already work, such as your team chat tool or intranet.
  4. Answers that always cite their sources with links.
  5. A clear “I couldn’t find this” response instead of a guess.
  6. A thumbs up or down button with an optional comment.

Skip file uploads, multi-step agents and fancy dashboards until the basics are trusted.

Step 6: Test before anyone relies on it

Run your list of real questions through the assistant and grade every answer. A simple three-level scale works: correct and well sourced, partially correct, or wrong or unsupported. Pay special attention to:

  • Questions where the answer changed recently.
  • Questions that should be refused because of permissions.
  • Questions with no answer in the content.
  • Ambiguous questions that need a follow-up.

Repeat this test every time you change the prompt, the model, the chunking or the sources. Without it, you cannot tell whether a change made things better or worse.

Step 7: Roll out and keep it alive

Launch to a small pilot group, gather feedback for a few weeks, then widen access. After launch, the work shifts from building to maintaining:

  • Review negative feedback weekly and fix the content, not just the prompt.
  • Track unanswered questions; they reveal documentation gaps.
  • Keep content owners responsible for their sections.
  • Watch usage costs and response times as adoption grows.

Common internal AI assistant pitfalls

  • Indexing everything at once and drowning good content in noise.
  • Treating permissions as a later phase.
  • Measuring success by usage alone, rather than by answer quality and time saved.
  • No named owner once the project team moves on.
  • Promising that the assistant “knows everything” instead of setting clear expectations about what it covers.

When to expand

Once the first team trusts the assistant, expansion is mostly a repeat of the same steps: a new audience, a new list of real questions, a content audit and a permissions review. Resist the temptation to simply connect every remaining source. Each new area should earn its place with a clear set of questions and an owner who keeps the content current. You can also start adding small actions, such as creating a ticket or drafting a document from a template, but treat each one as a separate feature with its own testing. This steady, team-by-team approach keeps quality high and makes problems easy to trace back to their source.

How SiteWright Technologies can build it with you

We build internal assistants as proper software: authenticated, permission-aware, tested and connected to the systems your team uses. If you want a partner to scope and deliver yours, take a look at our custom software development services or start a conversation with us.