Model selection and routing
We test candidate models on your data and route each step to the cheapest one that meets the accuracy bar, with fallbacks across providers.
Discipline02 · LLM Engineering
In shortUpdated 30 Oct 2026 · Labs 24 FDE Squad
LLM development at Labs 24, in short
LLM engineering is the discipline of turning large language models into reliable production software: choosing models, designing prompts and context, enforcing structured outputs, fine-tuning, and controlling cost, latency and quality. Labs 24 does LLM development with forward deployed engineers who build on your real data and measure every change against an eval set.
- What we build
- Extraction, classification, drafting, summarising, search and copilot features
- Model choice
- Per step, tested on your data: frontier APIs, open-weight or fine-tuned small models
- Cost
- Cost per task tracked; routing, caching and fine-tuning used to bring it down
- Privacy
- Private cloud endpoints or self-hosted open-weight models when data cannot leave
- Stack
- Claude, GPT, Gemini, Llama, Qwen, Mistral, vLLM, Ollama, Langfuse, eval suites
- Pricing
- Set after an initial embed and baseline; tied to measured outcomes
01 / What it means
The demo is easy. Production is the work.
Any team can get an LLM to give a good answer once. LLM engineering is making it give the right answer on the ten-thousandth messy input, in the format your system expects, at a cost that still makes sense at volume.
Every step gets its own model, prompt and testLLM feature built as a demo
- One big prompt to the biggest model
- Free-text output parsed with hope
- Judged on ten hand-picked examples
- Cost found out from the first invoice
LLM pipeline built forward deployed
- Work split into steps, each with the right model
- Schema-checked structured outputs
- Eval set built from hundreds of your real cases
- Cost and latency per task tracked from day one
- Fallbacks when a provider is slow or down
02 / Capabilities
LLM development capabilities.
Eight parts of LLM engineering we own end to end. Evals and retrieval get their own disciplines in the squad.
Prompt and context engineering
Versioned prompts, few-shot examples chosen from real cases, and context windows packed with only what the step needs.
Structured outputs
JSON schemas, typed tool calls and validation with automatic retry, so downstream systems get clean data every time.
Fine-tuning and distillation
LoRA fine-tunes and distillation from frontier models onto small ones, when volume and a fixed format make it pay.
Cost and latency engineering
Prompt caching, batching, streaming, shorter outputs and smaller models where they hold up. Every change measured against the baseline.
Private and self-hosted models
Private cloud endpoints, or open-weight models served on your own hardware with vLLM or Ollama when data must stay inside.
Documents, vision and voice
Reading scanned forms, invoices and photos, and handling speech in and out, with checks on every extracted field.
Safety and output checks
PII redaction, prompt-injection defences, content filters and rule checks on outputs before anything reaches a customer.
03 / Cost per task
Same accuracy. A quarter of the cost.
An example from an invoice-processing pipeline. Switch between sending every step to one frontier model and routing each step to the model that fits it.
StepModelCost per invoice
Classify the documentSmall model
Extract 24 fields to JSONFine-tuned small model
Match to purchase orderPlain code, no model
Explain a mismatch for reviewFrontier model
Summarise for the approverSmall model + cache
Cost per invoice$0.027
At 50,000 invoices / month$1,370
Field accuracy on eval set98.1%
Time per invoice3.1s
Example figures from one eval run. Your numbers depend on document mix, volume and model prices, which is why we measure them first.
04 / Choosing an approach
Prompt, retrieve, fine-tune or train?
Most problems are solved by the first two. We move down the list only when the eval numbers say we should.
01
Prompt and context
The rules fit in the prompt and a general model already knows the domain.
02
Retrieval (RAG)
The answer depends on your documents or data, and that knowledge changes often.
03
Fine-tune or distil
High volume, fixed format, and you need a smaller, faster or cheaper model to do it.
04
Train from scratch
Almost never worth it for a business problem. We will tell you so.
| Where the model runs | Data privacy | Quality | Cost at volume | Best for |
|---|---|---|---|---|
| Frontier API | Provider terms; no training on your data on business plans | Highest | Highest per call | Hard reasoning steps, low to medium volume |
| Private cloud endpoint | Inside your cloud account and region | High | Medium | Regulated data with frontier-level needs |
| Self-hosted open-weight | Never leaves your network | Good, higher when fine-tuned | Lowest per call, you run the hardware | High volume, strict data rules, on-prem |
05 / How it ships
Baseline first. Then make it cheaper.
We get the quality right on a strong model, then bring cost and latency down without letting accuracy slip. A demo every Friday, on your real data.
Every change is checked against your eval setStep 01 · Baseline
Collect real cases
We gather real inputs and correct answers from your team and turn them into an eval set.
- Eval set from your data
- Accuracy target agreed
- Current cost and time measured
Step 02 · Quality
Hit the accuracy bar
Prompts, structured outputs and a strong model working end to end on your real cases.
- Pipeline split into steps
- Schema-checked outputs
- Friday demo
Step 03 · Optimise
Cut cost and latency
Route steps to smaller or fine-tuned models, add caching and batching, and prove accuracy holds.
- Model routing per step
- Cost per task dashboard
- Regression evals
Step 04 · Production
Live with guardrails
Production traffic with monitoring, provider fallbacks and alerts when quality or cost drifts.
- Fallbacks across providers
- Drift and spend alerts
- Runbook and handover
06 / Stack
LLM models and tooling.
No model loyalty. We pick on your eval numbers and keep the design portable, so switching models later is a config change.
Models
ClaudeGPTGeminiLlamaQwenMistral
Serving and tuning
vLLMOllamaLoRA fine-tuningPrompt cachingBatch APIs
Quality and cost
Eval suitesLangfuseJSON schemaCost dashboardsAlerting
Data that cannot leave your network? Open-weight models run on your own hardware. See Production & Cloud Engineering.
07 / In production
LLM systems in production.
Under NDA, so clients stay unnamed. Each one runs on the client's own infrastructure with full IP transfer.
Law practice document assistant
Reads and checks case documents on self-hosted open-weight models, so client files never leave the firm.
Self-hosted LLM
Digital banking offer engine
Matches offers to customers and writes the offer copy, with every output checked against the bank's eligibility rules.
Structured outputs
D2C ecommerce operations stack
Catalogue enrichment and support replies for a D2C brand, routed across small and frontier models by task.
Model routing
08 / The squad around it
A model is one part of the system.
LLM engineers work beside the people who build the agents, feed the data, run the infrastructure and sign off the risk.
01
Agent Engineering
Agents that use these models to finish whole tasks.
03Data & Retrieval
RAG pipelines and the documents models reason over.
04Evals & Reliability
Eval sets, regression tests and drift monitoring.
05Production & Cloud
GPU serving, scaling and self-hosted models.
10Architecture
Reference architecture and build-or-buy calls.
11Security & Compliance
PII handling, data residency and audit trails.
09 / FAQ
LLM development FAQ.
What is LLM engineering?
LLM engineering is the discipline of turning large language models into reliable production software. It covers choosing models, designing prompts and context, enforcing structured outputs, fine-tuning, and controlling cost, latency and quality with evals. Labs 24 does it with forward deployed engineers who work on the client's real data.
What do LLM development services include?
LLM development services cover model selection and routing, prompt and context engineering, structured outputs and tool calling, fine-tuning and distillation, cost and latency optimisation, private or self-hosted deployment, document and vision handling, and safety checks. Labs 24 delivers all of these, measured against an eval set built from the client's own cases.
Should we fine-tune an LLM or use RAG?
Use retrieval (RAG) when the answer depends on documents or data that change often. Fine-tune when the task is high volume with a fixed format and you need a smaller, faster or cheaper model. Most projects start with prompting and RAG; Labs 24 fine-tunes only when eval results show it pays.
Which LLM is best for business use?
There is no single best model. Labs 24 tests candidate models such as Claude, GPT, Gemini, Llama, Qwen and Mistral on the client's own data and picks per step: frontier models for hard reasoning, small or fine-tuned models for high-volume extraction and classification.
Can we run open-source LLMs privately?
Yes. Labs 24 deploys open-weight models such as Llama, Qwen or Mistral on the client's own hardware or private cloud using vLLM or Ollama, so data never leaves the network. Fine-tuning can close much of the quality gap with frontier models for a specific task.
How do you reduce LLM costs?
By routing each step to the cheapest model that meets the accuracy bar, replacing model calls with plain code where possible, prompt caching, batching, shorter outputs, and fine-tuning small models for high-volume steps. Every change is checked against the eval set so accuracy does not drop.
How do you measure LLM quality?
With an eval set built from hundreds of the client's real inputs and correct answers, scored automatically on every change to prompts, models or code. Production outputs are traced and sampled so drift is caught early.
What are the steps in an LLM development project?
Labs 24 builds an eval set from real cases and measures the current baseline, gets a strong model to the accuracy target, then cuts cost and latency with routing, caching or fine-tuning, and finally moves production traffic over with monitoring and provider fallbacks. There is a demo every Friday throughout.
How much do LLM development services cost?
Labs 24 prices after an initial embed, once the current process and costs are baselined. Pricing is tied to measured outcomes such as accuracy and cost per task, and the client keeps full IP.