Source connectors and ingestion
Pipelines from SharePoint, Drive, Confluence, ticketing, email, ERP and CRM, with incremental sync and deletions handled.
Discipline03 · Data & Retrieval
In shortUpdated 30 Oct 2026 · Labs 24 FDE Squad
RAG development at Labs 24, in short
Retrieval-augmented generation (RAG) lets a language model answer from your own documents and data instead of its training. Labs 24 builds production RAG with forward deployed engineers: clean ingestion, structure-aware chunking, hybrid search, reranking, permission checks and citations, all scored against real questions from your team.
- What we build
- Knowledge assistants, document Q&A, policy and contract search, support answer engines
- Sources
- PDFs and scans, SharePoint, Drive, Confluence, tickets, email, ERP and CRM records
- Quality
- Retrieval hit rate and cited-answer rate measured on every change
- Security
- Source permissions mirrored into the index and checked on every query
- Stack
- Qdrant, Pinecone, Postgres with pgvector on Neon, LlamaIndex, keyword search, rerankers
- Pricing
- Set after an initial embed and source audit
01 / What it means
Most RAG problems are data problems.
When a RAG system gets an answer wrong, the model is rarely the cause. The right page was never parsed, the clause was split in half, an old version ranked first, or the search missed the word the document actually uses. We fix the data path first.
Every answer links back to the page it came fromRAG built as a demo
- Documents split every 500 tokens
- Vector search only
- Old and new versions side by side
- Anyone can retrieve anything
RAG built forward deployed
- Chunks follow headings, clauses and tables
- Keyword and vector search, then a reranker
- Version, date and owner on every chunk
- Source permissions checked on every query
- Every answer cites the page it used
02 / Capabilities
RAG development capabilities.
Eight parts of data and retrieval engineering we own end to end, from the source system to the cited answer.
Document parsing
Layout-aware parsing of PDFs, scans, slides and spreadsheets, with OCR where needed and tables kept as tables.
Chunking and enrichment
Chunks that follow headings, clauses and sections, tagged with source, version, date, owner and document type.
Hybrid search and reranking
Keyword and vector search combined, filtered by metadata, then reranked so the best passage comes first.
Permission-aware retrieval
Access rules from the source systems mirrored into the index and enforced on every query, so nobody sees what they shouldn't.
Grounded answers with citations
Answers built only from retrieved passages, with links to the exact page or record, and a clear "not found" when the data isn't there.
Questions over structured data
Text-to-SQL and API lookups for questions that live in databases and spreadsheets, checked before they run.
Retrieval evals
A question set with known answers from your team, scored for retrieval hit rate, faithfulness and citation accuracy on every change.
03 / Retrieval, side by side
Same question. Different retrieval.
An example from a contracts assistant. Switch between basic vector search and a production retrieval pipeline, and see what the model is handed.
"What is the late-delivery penalty in our 2026 contract with Northwind Freight?"
Vector searchKeyword searchMetadata filterPermission checkReranker
Retrieval hit rate (top 5)94%
Answers with valid citations97%
Wrong-version answers0.4%
Time to answer1.6s
Example figures from one eval run on a 300-question set. Your numbers depend on your documents, which is why we build the question set first.
04 / Where RAG goes wrong
Why RAG gives wrong answers, and the fix.
The data path has eight steps. A failure at any one of them looks the same to the user: a confident, wrong answer.
- ConnectSync sources
- ParseText, tables, scans
- ChunkBy structure
- EnrichVersion, date, owner
- IndexKeyword + vectors
- RetrieveFilter + permissions
- RerankBest passage first
- AnswerWith citations
| What goes wrong | What users see | What we do |
|---|---|---|
| Chunks split mid-clause | Half answers that miss the exceptions | Chunk by heading, clause and table, not by token count |
| Old versions retrieved | Confident answers from superseded documents | Version and date on every chunk, filters, and sync that removes deleted files |
| Scans and tables lost | "Not found" for data that is in the PDF | Layout-aware parsing and OCR, tables stored as tables |
| Words don't match | Search for "late fee" misses "liquidated damages" | Hybrid keyword and vector search, then a reranker |
| Permission leaks | People see documents they shouldn't | Source permissions mirrored into the index and checked per query |
| No measurement | Quality judged by anecdotes | A question set with known answers, scored on every change |
05 / How it ships
Questions first. Then the index.
We start from the questions your team asks and the answers they expect, then build the data path to get them right. A demo every Friday, on your real documents.
Your team writes the questions. We make the answers right.Step 01 · Audit
Sources and questions
We map where the knowledge lives and collect real questions with correct answers from your team.
- Source and permission map
- Question set with answers
- Baseline of today's search
Step 02 · Index
First cited answers
Ingestion, parsing and chunking for the main sources, with cited answers on real questions.
- Connectors and parsing
- Hybrid index
- Friday demo
Step 03 · Tune
Raise the hit rate
Reranking, metadata filters and permission checks, tuned until the question set scores where it should.
- Reranker and filters
- Permission checks
- Eval scores per change
Step 04 · Production
Live and in sync
Scheduled sync, monitoring and a feedback button, so wrong answers become new test cases.
- Sync and deletion jobs
- Answer feedback loop
- Runbook and handover
06 / Stack
RAG stack and vector databases.
We pick the store for your scale and your infrastructure. Often Postgres you already run is enough.
Search and storage
QdrantPineconePostgres + pgvectorNeonKeyword search
Pipelines
LlamaIndexTemporaln8nOCR and layout parsingRerankers
Quality
Question setsRetrieval evalsLangfuseFeedback loop
Documents that cannot leave your network? The whole pipeline, including embeddings and models, can run on your own hardware. See Production & Cloud Engineering.
07 / In production
RAG systems in production.
Under NDA, so clients stay unnamed. Each one runs on the client's own infrastructure with full IP transfer.
Law practice document assistant
Answers questions across case files with citations to the page, on self-hosted models so client documents never leave the firm.
Cited answers
Rubber manufacturing ops tool
Lets the shop floor search SOPs, specs and batch records in plain language, with answers tied to the current document version.
Versioned sources
Clinic chain follow-up assistant
Pulls the right care instructions and clinic policies for each follow-up, with staff access rules enforced on every lookup.
Permission-aware
08 / The squad around it
Retrieval feeds everything else.
The same index serves the models, the agents and the people searching. These disciplines work on it together.
01
Agent Engineering
Agents that look things up before they act.
02LLM Engineering
The models that turn retrieved passages into answers.
04Evals & Reliability
Question sets, scoring and drift alerts.
05Production & Cloud
Index hosting, scaling and self-hosted embeddings.
06Integration
The ERP, CRM and document systems we pull from.
11Security & Compliance
Access control, data residency and audit trails.
09 / FAQ
RAG development FAQ.
What is RAG (retrieval-augmented generation)?
RAG is a way of letting a language model answer from your own documents and data. A retrieval step finds the most relevant passages for each question, and the model writes an answer using only those passages, with citations back to the source.
What do RAG development services include?
RAG development covers connecting to source systems, parsing documents and scans, structure-aware chunking, metadata enrichment, hybrid keyword and vector search, reranking, permission-aware retrieval, cited answers and retrieval evals. Labs 24 delivers all of these with forward deployed engineers and scores them against real questions from the client's team.
Why do RAG systems give wrong answers?
Usually because of the data path, not the model: pages that were never parsed, clauses split across chunks, old document versions ranking first, or searches that miss the words a document actually uses. Labs 24 fixes these with layout-aware parsing, structure-aware chunking, version metadata, hybrid search and reranking.
Should we use RAG or fine-tuning?
Use RAG when answers depend on documents or data that change and must be cited. Use fine-tuning when you need a model to follow a fixed format or style at high volume. Many production systems use both: RAG for knowledge, a fine-tuned model for format.
Which vector database should we use?
It depends on scale and where you want the data to live. Postgres with pgvector, for example on Neon, is often enough and keeps data in a database you already run. Qdrant and Pinecone suit larger indexes and heavier filtering. Labs 24 picks after measuring retrieval on your own question set.
Can a RAG system respect document permissions?
Yes. Labs 24 mirrors access rules from the source systems, such as SharePoint or Drive permissions, into the index and checks them on every query, so each user only retrieves documents they are allowed to see.
Can RAG answer questions over spreadsheets and databases?
Yes, through text-to-SQL or API lookups instead of plain text search. Queries run with read-only access and are checked before execution, and answers cite the records they came from.
What are the steps in a RAG project?
Labs 24 maps the sources and collects real questions with correct answers, builds ingestion and a hybrid index with cited answers, tunes reranking, filters and permissions until the question set scores well, then goes live with scheduled sync, monitoring and a feedback loop.
How much does RAG development cost?
Labs 24 prices after an initial embed and source audit, once the number of sources, documents and users is known. Clients keep full IP, and the index runs in their own infrastructure.