Data & analytics
Knowledge Base
Collections of documents an agent can search by meaning and quote with citations. This is how you stop an agent inventing your policies.
Open Data → Knowledge Base. A collection is a named group of documents; agents attach to collections, not to individual files.
Adding sources
Add source covers one-shot ingestion — files, a web page, a repository. Connect links an external service that is synced on a schedule and kept deduplicated. Both land documents in the same collection and the same retrieval pipeline.
File upload
Accepted extensions, exactly:
.txt .md .markdown .csv .tsv .log
.html .htm .xml .yaml .yml .json
.rtf .pdf .docxA scanned PDF yields nothing
Web page / crawl
Give a URL and the page is fetched and converted to text. Use it for public documentation and policy pages. Pages behind a login cannot be fetched.
GitHub repository
Ingests source and docs from a repository so an agent can answer questions about a codebase.
Connected services — Drive, Notion, SharePoint, Dropbox
Connect opens a wizard for four providers. Credentials are pasted tokens (the platform's BYOK pattern — no OAuth consent screens to register), validated against the provider at save time, encrypted at rest, and never sent back to the browser: editing a source shows empty credential fields, and leaving them empty keeps what is stored.
| Provider | Credentials | What syncs | Mirrors sharing? |
|---|---|---|---|
| Google Drive | Access token, or refresh token + OAuth client pair for unattended syncs | A folder (subfolders to depth 5): Google Docs/Sheets/Slides exported as text, plus text-format files | Yes — per-file permissions |
| Notion | Internal-integration secret; share the pages with the integration | Listed page IDs and every page of listed databases | No — the API exposes none |
| SharePoint | Entra app registration: tenant + client ID + secret (Files.Read.All, admin-consented) | A document library or folder path, text-format files | Yes — per-item permissions |
| Dropbox | Access token, or refresh token + app key/secret for unattended syncs | A folder path (or everything); native content hashes make change detection exact | Yes — file members, best-effort |
Per source, listings cap at 500 items and each item's text at 400k characters. Anything the connector saw but did not ingest — an unsupported binary, a too-deep folder, the cap — is listed on the source card with its reason, never silently dropped.
Short-lived tokens and schedules don't mix
Scheduled sync — indexing without duplicates
Each connected source has a schedule: manual, hourly, daily or weekly. Scheduled syncs run on the platform's maintenance pass (the same engine that refreshes BI and SaaS data), and a claim on the source's next-run time guarantees that multiple app instances never sync the same source twice.
Change detection is two-level, and both levels exist to make an hourly schedule cheap:
- 1
Version skip — unchanged files are not downloaded
The provider's change marker (modified time, revision, native hash) is stamped on each document. Unchanged marker ⇒ the item is skipped without being downloaded — a 400-file folder re-syncs for the price of a listing. - 2
Content skip — unchanged text is not re-embedded
Providers bump modified times on moves, permission edits and comments. Downloaded text is hashed (sha256); if it matches what is stored, the marker is refreshed and the document is not re-chunked or re-embedded. Embedding spend follows actual content change, nothing else.
Files deleted at the provider delete their documents (and chunks) here. The source card shows each sync's outcome as +added ~updated =unchanged −removed, and a database uniqueness constraint on (source, remote item) makes duplicate documents impossible even if everything above were wrong.
Sync statuses
ok is a clean pass. error names the provider's refusal verbatim — a revoked token reads "Dropbox 401: invalid_access_token", not "0 documents". embed failed means documents were saved but semantic indexing didn't finish: retrieval falls back to keyword search for them until a re-sync succeeds, and the owner gets a notification either way.What happens on ingest
file ──▶ extract text ──▶ split into chunks ──▶ embed each chunk ──▶ store
(few hundred words, (vector = position
overlapping) in meaning-space)
question ──▶ embed ──▶ nearest chunks ──▶ pasted into the prompt as [1] [2] …Chunking matters more than people expect. Retrieval returns chunks, not documents — so if the sentence answering a question is split across two chunks, neither answers it well. Chunks overlap slightly to soften this.
Why it works this way
Which model does the embedding
Set this per collection under RAG settings → Embedding. OpenRouter is the default — either through your own integration or, with no integration at all, through the operator's OPENROUTER_API_KEY. That keeps embedding off the OpenAI quota that chat, document generation and retrieval already share; when that quota runs out, knowledge-base search would otherwise go down with it. The operator's OpenAI key is the fallback, and any other connected provider exposing an OpenAI-compatible /embeddings endpoint can be selected instead.
Changing the model means re-embedding
Note
text-embedding-3-* models truncate to any size on request. If a model returns a different width the embed fails with a message saying so rather than writing unusable vectors.The OpenRouter default is openai/text-embedding-3-small because it is the same 1536-d space the operator's OpenAI key produces: moving a collection onto OpenRouter to get off an exhausted OpenAI quota costs nothing and leaves existing chunks searchable. The other options are different spaces, so choosing one means re-indexing.
| Model | Native width | Notes |
|---|---|---|
openai/text-embedding-3-small | 1536 | Default. Same vector space as the built-in OpenAI key. |
openai/text-embedding-3-large | 3072 | Truncates to 1536 on request. |
google/gemini-embedding-001 | 3072 | Truncates to 1536 on request. |
qwen/qwen3-embedding-8b | 4096 | Truncates to 1536 on request. |
qwen/qwen3-embedding-4b | 2560 | Truncates to 1536 on request. |
Every model in that list was called against OpenRouter's live endpoint and confirmed to return 1536 dimensions. That check is not ceremony: OpenRouter does not list embedding models in its public /models catalogue, so a plausible-looking id is no evidence the model exists. Two NVIDIA nemotron ids used to be offered here and both returned 404 No endpoints found — selecting one produced a failed embed with nothing to indicate the model had never been available.
Chunking modes
Retrieval and generation want opposite things from a chunk. Matching is most precise when chunks are small and about one idea; answering is best when the model can see the whole passage. RAG Settings → Chunking → Chunking Mode decides how that tension is resolved for a document.
| Mode | What is embedded | What the model reads | Use it when |
|---|---|---|---|
| Flat | The chunk | The same chunk | Short documents, FAQs, anything where one chunk is already a complete thought. This is the default and was the only behaviour before. |
| Parent-child | Small child chunks | The child's parent | Long reference material — manuals, contracts, policies — where the sentence that matches is meaningless without the section around it. |
| Q&A | A generated question | The question and its answer | Support content and policy documents that people query in natural questions. Costs one model call per passage at index time. |
Parent-child sets two sizes: the parent is what reaches the model, and the existing chunk size becomes the child. A child is capped at half the parent, because a child the same size as its parent is flat chunking with extra bookkeeping. Parents do not overlap each other; children overlap within a parent.
Q&A exists because a question and a statement are different kinds of text, and that difference is a real part of the distance between their vectors. Asking “How do I rotate a key?” against a paragraph that says “Rotation issues a replacement…” is a harder match than asking it against the generated question “How do I rotate a key?”. It needs OPENROUTER_API_KEY; if generation fails, the run reports it rather than quietly writing flat chunks, so a collection never disagrees with the mode shown in its own settings.
Changing the mode does not rewrite existing chunks
Hybrid search and weighting
Vector search finds meaning and blurs exact strings; an error code, a part number or a surname is exactly the kind of token embeddings smooth away. Keyword search is the opposite. RAG Settings → Retrieval sets which of them runs, per knowledge base.
| Mode | What runs |
|---|---|
| Semantic | Vector search only. The default, and what every collection did before this existed. |
| Hybrid | Vector and Postgres full-text search over the same chunks, merged by weight. |
| Keyword | Full-text search only. |
The weighting slider splits the score between them. Each retriever’s scores are normalised within its own list first, because cosine similarity (roughly 0.3–0.9) and text rank (roughly 0.0–0.3) are not comparable numbers — added raw, the slider would do almost nothing across most of its range. A chunk found by bothretrievers scores above one found by only one, which is usually the result you want.
Changing retrieval mode takes effect immediately and needs no re-embedding: it changes how the existing index is queried, not how it was built. Note that keyword search also indexes the generated question on Q&A rows, because the answer text often does not contain the words someone would search for.
Retrieval settings — the real numbers
| Setting | Default | Range | Notes |
|---|---|---|---|
| top-K | 5 | 1 – 8 (hard cap) | How many chunks are retrieved and pasted into the prompt. Asking for more than 8 is clamped. |
| Candidate pool with a reranker | 3 × top-K, max 20 | — | With a reranker configured, a wider first pass is fetched and then re-scored down to top-K. This is where the accuracy gain comes from. |
| Snippet radius | 280 characters | — | How much text either side of a match is shown in the citation snippet under the answer. |
Reranking
Configured per agent on the Knowledge tab: a Provider and a Re-rank model (for example llama-nemotron-rerank-vl-1b-v2).
- Cost — one extra model call per retrieval.
- When it pays — collections with many near-identical passages: long contracts, several revisions of one policy, product manuals for a family of similar products.
- When it doesn't — a small collection of clearly distinct documents. The first pass is already right.
Attaching a collection to an agent
- 1
Create the collection and ingest into it
Keep unrelated subject matter in separate collections — a mixed collection retrieves measurably worse. - 2
Agent Builder → Knowledge → link it
Linking auto-enables thekb_searchtool. - 3
Add the grounding instruction to the system prompt
Without this the model happily falls back on general knowledge and you will not notice. - 4
Turn on Citation Check
Guardrails tab. It flags an answer that cites nothing when sources were available — see Guardrails.
Answer only from the provided sources. Cite them inline as [1], [2].
If the sources do not contain the answer, reply exactly:
"I don't have that in my documentation."
Never fill a gap with general knowledge.Retrieval happens automatically before each turn, with numbered citations inserted. The kb_search tool additionally lets the agent search on demand, mid-answer, when its first read wasn't enough.
Graph search
Ordinary retrieval finds chunks resembling the question. Graph search also follows relationships extracted between entities across documents, answering questions plain retrieval structurally cannot — "which suppliers are affected by the clause in appendix B?", where no single chunk contains both halves.
- 1
Build the graph
Knowledge → Graph, on the collection. This is a separate, slower pass over the documents. - 2
Enable the tool
Agent Builder → Tools →kb_graph_search.
Careful
kb_graph_search without building the graph first returns nothing — and the agent will treat "nothing" as "no information exists" rather than as a configuration problem.Sharing and access
Collections are private to their owner. An administrator can grant a user or group read-only access from Access control; shared collections show a Shared badge and cannot be edited by the recipient.
Because the grant is enforced in the database, an agent's retrieval inherits it automatically — there is no second permission to keep in sync.
Per-source access scopes
Connected sources add a second, finer layer: within a collection someone can already see, which of its synced documents may they retrieve? Each connected source picks one of three scopes, enforced at retrieval — vector and keyword paths alike, before any reranking model sees the text:
| Scope | Who retrieves the documents |
|---|---|
| Everyone with this KB | Default. Documents behave exactly like uploads — collection visibility decides. Every pre-existing document works this way. |
| Only me | The connecting user, full stop — even when the collection itself is shared or granted. |
| Match source permissions | Sharing is mirrored from the provider per document: people shared on the original file (by email or domain, at Drive/SharePoint/Dropbox) retrieve it here; everyone else doesn't. A publicly-linked file stays public. |
- Owner always retrieves their own documents, whatever the scope — a restriction you configured cannot lock you out of your own data.
- Fails toward deny. A provider that exposes no sharing info (Notion's API does not; a Dropbox plan without member listing) leaves documents owner-only, and the sync stats count them so the choice is visible. Tenant-wide links ("anyone in the organisation") match nobody but the owner — tenant membership can't be verified from here, and a wrong deny is recoverable where a wrong allow is not.
- Public embeds are anonymous. An embedded assistant retrieves only default-scope documents (and provider-public ones) — never "Only me" or ACL-restricted material, even though the embed runs under its owner's account.
When an answer is wrong
Work through these in order. The cause is nearly always one of them.
| # | Check | What it means |
|---|---|---|
| 1 | Does the document show a non-zero chunk count? | Zero means extraction produced nothing — almost always a scanned PDF. |
| 2 | Open the run in Traces: was anything retrieved? | Empty means the collection isn't linked to this agent, or the query embedded far from everything in it. |
| 3 | Are the retrieved chunks on-topic but not the right passage? | Chunking or phrasing. Try a reranker; consider re-uploading a better-structured source. |
| 4 | Retrieved correctly but ignored? | The system prompt doesn't require grounding, or another tool's result was more prominent. |
| 5 | Are knowledge sources listed under the answer? | They appear when the answer cites them, or when nothing else grounded it. If a web search answered instead, you will see links — that is correct, not a bug. |
Getting better results
- Prefer structured documents. Headings give the splitter meaningful boundaries; an unbroken wall of text does not.
- Split by subject, not by department. One collection per coherent topic retrieves better than one collection of everything.
- Delete superseded versions. Three revisions of one policy retrieve interchangeably and the agent has no way to know which is current. This is the single most common cause of confidently outdated answers.
- Put numbers in tables. If the question is really arithmetic it belongs in Data Catalog — retrieval quotes prose, it does not compute.
- Name documents descriptively. The document name appears in the citation a reader sees, so
refund-policy-2026.pdfbeatsfinal_v3.pdf.