Data & analytics

Knowledge Base

Collections of documents an agent can search by meaning and quote with citations. This is how you stop an agent inventing your policies.

Open Data → Knowledge Base. A collection is a named group of documents; agents attach to collections, not to individual files.

Adding sources

Add source covers one-shot ingestion — files, a web page, a repository. Connect links an external service that is synced on a schedule and kept deduplicated. Both land documents in the same collection and the same retrieval pipeline.

File upload

Accepted extensions, exactly:

Accepted file types
.txt   .md   .markdown   .csv   .tsv   .log
.html  .htm  .xml        .yaml  .yml  .json
.rtf   .pdf  .docx

A scanned PDF yields nothing

Text is extracted, not OCR'd. A PDF that is images of pages produces zero chunks and the agent will answer from general knowledge with no sign anything is wrong. After uploading, check the document shows a non-zero chunk count — that is the one-second test that catches this.

Web page / crawl

Give a URL and the page is fetched and converted to text. Use it for public documentation and policy pages. Pages behind a login cannot be fetched.

GitHub repository

Ingests source and docs from a repository so an agent can answer questions about a codebase.

Connected services — Drive, Notion, SharePoint, Dropbox

Connect opens a wizard for four providers. Credentials are pasted tokens (the platform's BYOK pattern — no OAuth consent screens to register), validated against the provider at save time, encrypted at rest, and never sent back to the browser: editing a source shows empty credential fields, and leaving them empty keeps what is stored.

ProviderCredentialsWhat syncsMirrors sharing?
Google DriveAccess token, or refresh token + OAuth client pair for unattended syncsA folder (subfolders to depth 5): Google Docs/Sheets/Slides exported as text, plus text-format filesYes — per-file permissions
NotionInternal-integration secret; share the pages with the integrationListed page IDs and every page of listed databasesNo — the API exposes none
SharePointEntra app registration: tenant + client ID + secret (Files.Read.All, admin-consented)A document library or folder path, text-format filesYes — per-item permissions
DropboxAccess token, or refresh token + app key/secret for unattended syncsA folder path (or everything); native content hashes make change detection exactYes — file members, best-effort

Per source, listings cap at 500 items and each item's text at 400k characters. Anything the connector saw but did not ingest — an unsupported binary, a too-deep folder, the cap — is listed on the source card with its reason, never silently dropped.

Short-lived tokens and schedules don't mix

A pasted access token is fine for a first manual sync, but Google's expire in about an hour and Dropbox's eventually rotate — a scheduled sync running at 3am needs the refresh-token form. The wizard says which fields serve which purpose.

Scheduled sync — indexing without duplicates

Each connected source has a schedule: manual, hourly, daily or weekly. Scheduled syncs run on the platform's maintenance pass (the same engine that refreshes BI and SaaS data), and a claim on the source's next-run time guarantees that multiple app instances never sync the same source twice.

Change detection is two-level, and both levels exist to make an hourly schedule cheap:

  1. 1

    Version skip — unchanged files are not downloaded

    The provider's change marker (modified time, revision, native hash) is stamped on each document. Unchanged marker ⇒ the item is skipped without being downloaded — a 400-file folder re-syncs for the price of a listing.
  2. 2

    Content skip — unchanged text is not re-embedded

    Providers bump modified times on moves, permission edits and comments. Downloaded text is hashed (sha256); if it matches what is stored, the marker is refreshed and the document is not re-chunked or re-embedded. Embedding spend follows actual content change, nothing else.

Files deleted at the provider delete their documents (and chunks) here. The source card shows each sync's outcome as +added ~updated =unchanged −removed, and a database uniqueness constraint on (source, remote item) makes duplicate documents impossible even if everything above were wrong.

Sync statuses

ok is a clean pass. error names the provider's refusal verbatim — a revoked token reads "Dropbox 401: invalid_access_token", not "0 documents". embed failed means documents were saved but semantic indexing didn't finish: retrieval falls back to keyword search for them until a re-sync succeeds, and the owner gets a notification either way.

What happens on ingest

file ──▶ extract text ──▶ split into chunks ──▶ embed each chunk ──▶ store
                             (few hundred words,      (vector = position
                              overlapping)             in meaning-space)

question ──▶ embed ──▶ nearest chunks ──▶ pasted into the prompt as [1] [2] …
Document to retrievable chunk.

Chunking matters more than people expect. Retrieval returns chunks, not documents — so if the sentence answering a question is split across two chunks, neither answers it well. Chunks overlap slightly to soften this.

Why it works this way

Embeddings match on meaning, not keywords: "how long do I have to send it back" finds a paragraph about the "30-day return window" with no shared words. The flip side is that meaning-similar but irrelevant text also scores well — which is why chunk boundaries and reranking matter.

Which model does the embedding

Set this per collection under RAG settings → Embedding. OpenRouter is the default — either through your own integration or, with no integration at all, through the operator's OPENROUTER_API_KEY. That keeps embedding off the OpenAI quota that chat, document generation and retrieval already share; when that quota runs out, knowledge-base search would otherwise go down with it. The operator's OpenAI key is the fallback, and any other connected provider exposing an OpenAI-compatible /embeddings endpoint can be selected instead.

Changing the model means re-embedding

Vectors from two different models are not comparable — searching model A's chunks with model B's query vector does not error, it quietly returns wrong matches. So the provider and model are recorded on each document when it is embedded, and the question is always embedded with whatever that document used. Switching a collection to a new model therefore only affects documents embedded from then on: use Back-fill embeddings to move the existing ones across.

Note

The vector store is fixed at 1536 dimensions. A model must be able to emit that width — the OpenAI text-embedding-3-* models truncate to any size on request. If a model returns a different width the embed fails with a message saying so rather than writing unusable vectors.

The OpenRouter default is openai/text-embedding-3-small because it is the same 1536-d space the operator's OpenAI key produces: moving a collection onto OpenRouter to get off an exhausted OpenAI quota costs nothing and leaves existing chunks searchable. The other options are different spaces, so choosing one means re-indexing.

ModelNative widthNotes
openai/text-embedding-3-small1536Default. Same vector space as the built-in OpenAI key.
openai/text-embedding-3-large3072Truncates to 1536 on request.
google/gemini-embedding-0013072Truncates to 1536 on request.
qwen/qwen3-embedding-8b4096Truncates to 1536 on request.
qwen/qwen3-embedding-4b2560Truncates to 1536 on request.

Every model in that list was called against OpenRouter's live endpoint and confirmed to return 1536 dimensions. That check is not ceremony: OpenRouter does not list embedding models in its public /models catalogue, so a plausible-looking id is no evidence the model exists. Two NVIDIA nemotron ids used to be offered here and both returned 404 No endpoints found — selecting one produced a failed embed with nothing to indicate the model had never been available.

Chunking modes

Retrieval and generation want opposite things from a chunk. Matching is most precise when chunks are small and about one idea; answering is best when the model can see the whole passage. RAG Settings → Chunking → Chunking Mode decides how that tension is resolved for a document.

ModeWhat is embeddedWhat the model readsUse it when
FlatThe chunkThe same chunkShort documents, FAQs, anything where one chunk is already a complete thought. This is the default and was the only behaviour before.
Parent-childSmall child chunksThe child's parentLong reference material — manuals, contracts, policies — where the sentence that matches is meaningless without the section around it.
Q&AA generated questionThe question and its answerSupport content and policy documents that people query in natural questions. Costs one model call per passage at index time.

Parent-child sets two sizes: the parent is what reaches the model, and the existing chunk size becomes the child. A child is capped at half the parent, because a child the same size as its parent is flat chunking with extra bookkeeping. Parents do not overlap each other; children overlap within a parent.

Q&A exists because a question and a statement are different kinds of text, and that difference is a real part of the distance between their vectors. Asking “How do I rotate a key?” against a paragraph that says “Rotation issues a replacement…” is a harder match than asking it against the generated question “How do I rotate a key?”. It needs OPENROUTER_API_KEY; if generation fails, the run reports it rather than quietly writing flat chunks, so a collection never disagrees with the mode shown in its own settings.

Changing the mode does not rewrite existing chunks

Re-chunking means paying to embed the whole document again, so it is never automatic. Use Re-index “…” with these settings in the Chunking tab to rebuild a collection under the new mode. Documents added afterwards use the new mode already.

Hybrid search and weighting

Vector search finds meaning and blurs exact strings; an error code, a part number or a surname is exactly the kind of token embeddings smooth away. Keyword search is the opposite. RAG Settings → Retrieval sets which of them runs, per knowledge base.

ModeWhat runs
SemanticVector search only. The default, and what every collection did before this existed.
HybridVector and Postgres full-text search over the same chunks, merged by weight.
KeywordFull-text search only.

The weighting slider splits the score between them. Each retriever’s scores are normalised within its own list first, because cosine similarity (roughly 0.3–0.9) and text rank (roughly 0.0–0.3) are not comparable numbers — added raw, the slider would do almost nothing across most of its range. A chunk found by bothretrievers scores above one found by only one, which is usually the result you want.

Changing retrieval mode takes effect immediately and needs no re-embedding: it changes how the existing index is queried, not how it was built. Note that keyword search also indexes the generated question on Q&A rows, because the answer text often does not contain the words someone would search for.

Retrieval settings — the real numbers

SettingDefaultRangeNotes
top-K51 – 8 (hard cap)How many chunks are retrieved and pasted into the prompt. Asking for more than 8 is clamped.
Candidate pool with a reranker3 × top-K, max 20With a reranker configured, a wider first pass is fetched and then re-scored down to top-K. This is where the accuracy gain comes from.
Snippet radius280 charactersHow much text either side of a match is shown in the citation snippet under the answer.

Reranking

Configured per agent on the Knowledge tab: a Provider and a Re-rank model (for example llama-nemotron-rerank-vl-1b-v2).

  • Cost — one extra model call per retrieval.
  • When it pays — collections with many near-identical passages: long contracts, several revisions of one policy, product manuals for a family of similar products.
  • When it doesn't — a small collection of clearly distinct documents. The first pass is already right.

Attaching a collection to an agent

  1. 1

    Create the collection and ingest into it

    Keep unrelated subject matter in separate collections — a mixed collection retrieves measurably worse.
  2. 2

    Agent Builder → Knowledge → link it

    Linking auto-enables the kb_search tool.
  3. 3

    Add the grounding instruction to the system prompt

    Without this the model happily falls back on general knowledge and you will not notice.
  4. 4

    Turn on Citation Check

    Guardrails tab. It flags an answer that cites nothing when sources were available — see Guardrails.
The grounding block, minimum viable
Answer only from the provided sources. Cite them inline as [1], [2].
If the sources do not contain the answer, reply exactly:
"I don't have that in my documentation."
Never fill a gap with general knowledge.

Retrieval happens automatically before each turn, with numbered citations inserted. The kb_search tool additionally lets the agent search on demand, mid-answer, when its first read wasn't enough.

Graph search

Ordinary retrieval finds chunks resembling the question. Graph search also follows relationships extracted between entities across documents, answering questions plain retrieval structurally cannot — "which suppliers are affected by the clause in appendix B?", where no single chunk contains both halves.

  1. 1

    Build the graph

    Knowledge → Graph, on the collection. This is a separate, slower pass over the documents.
  2. 2

    Enable the tool

    Agent Builder → Tools → kb_graph_search.

Careful

Enabling kb_graph_search without building the graph first returns nothing — and the agent will treat "nothing" as "no information exists" rather than as a configuration problem.

Sharing and access

Collections are private to their owner. An administrator can grant a user or group read-only access from Access control; shared collections show a Shared badge and cannot be edited by the recipient.

Because the grant is enforced in the database, an agent's retrieval inherits it automatically — there is no second permission to keep in sync.

Per-source access scopes

Connected sources add a second, finer layer: within a collection someone can already see, which of its synced documents may they retrieve? Each connected source picks one of three scopes, enforced at retrieval — vector and keyword paths alike, before any reranking model sees the text:

ScopeWho retrieves the documents
Everyone with this KBDefault. Documents behave exactly like uploads — collection visibility decides. Every pre-existing document works this way.
Only meThe connecting user, full stop — even when the collection itself is shared or granted.
Match source permissionsSharing is mirrored from the provider per document: people shared on the original file (by email or domain, at Drive/SharePoint/Dropbox) retrieve it here; everyone else doesn't. A publicly-linked file stays public.
  • Owner always retrieves their own documents, whatever the scope — a restriction you configured cannot lock you out of your own data.
  • Fails toward deny. A provider that exposes no sharing info (Notion's API does not; a Dropbox plan without member listing) leaves documents owner-only, and the sync stats count them so the choice is visible. Tenant-wide links ("anyone in the organisation") match nobody but the owner — tenant membership can't be verified from here, and a wrong deny is recoverable where a wrong allow is not.
  • Public embeds are anonymous. An embedded assistant retrieves only default-scope documents (and provider-public ones) — never "Only me" or ACL-restricted material, even though the embed runs under its owner's account.

When an answer is wrong

Work through these in order. The cause is nearly always one of them.

#CheckWhat it means
1Does the document show a non-zero chunk count?Zero means extraction produced nothing — almost always a scanned PDF.
2Open the run in Traces: was anything retrieved?Empty means the collection isn't linked to this agent, or the query embedded far from everything in it.
3Are the retrieved chunks on-topic but not the right passage?Chunking or phrasing. Try a reranker; consider re-uploading a better-structured source.
4Retrieved correctly but ignored?The system prompt doesn't require grounding, or another tool's result was more prominent.
5Are knowledge sources listed under the answer?They appear when the answer cites them, or when nothing else grounded it. If a web search answered instead, you will see links — that is correct, not a bug.

Getting better results

  • Prefer structured documents. Headings give the splitter meaningful boundaries; an unbroken wall of text does not.
  • Split by subject, not by department. One collection per coherent topic retrieves better than one collection of everything.
  • Delete superseded versions. Three revisions of one policy retrieve interchangeably and the agent has no way to know which is current. This is the single most common cause of confidently outdated answers.
  • Put numbers in tables. If the question is really arithmetic it belongs in Data Catalog — retrieval quotes prose, it does not compute.
  • Name documents descriptively. The document name appears in the citation a reader sees, so refund-policy-2026.pdf beats final_v3.pdf.