Reference architecture
A layered target design for your AI systems, from channels to systems of record, that every new use case can follow.
Discipline10 · Architecture
In shortUpdated 30 Oct 2026 · Labs 24 FDE Squad
AI architecture at Labs 24, in short
AI architecture is the design of how an AI system's parts fit together: channels, orchestration, models, retrieval, data, integrations, security and operations. Good AI architecture keeps each layer replaceable, so a new model, database or channel is a configuration change rather than a rebuild. Labs 24 designs it with forward deployed engineers, inside the client's environment.
- Principle
- Model-agnostic and layer-by-layer replaceable, with no single vendor holding the system
- Decisions
- Build or buy, cloud or on-prem, which models and stores, each recorded with its reasons
- Layers
- Channels, orchestration, models and retrieval, data and systems of record, operations
- Reviews
- Architecture reviews of existing pilots and vendor proposals
- Stack
- LangGraph, Temporal, n8n, Claude, GPT, Gemini, Llama, Qwen, Mistral, vLLM, Qdrant, Pinecone, Neon, Langfuse
- Output
- Target architecture, decision records, a proven riskiest piece and a roadmap
01 / What it means
The best model today won't stay the best model for long.
AI systems built around one vendor's model, one framework or one database age fast. Prices drop, better models arrive, regulators ask for data to move. Architecture decides whether that is a config change or a full rebuild.
Every decision written down, with its reasonsAI architecture by accident
- Provider SDK calls spread through the code
- Prompts, data and logic mixed together
- Bought a tool before mapping the need
- Nobody remembers why it was built this way
AI architecture by design
- One gateway between the code and every model
- Clear layers with stable interfaces
- Build or buy decided per component
- Decision records for every major choice
- Riskiest piece proven before the full build
02 / Capabilities
AI architecture capabilities.
Eight parts of AI architecture we own end to end, from the current-state map to the decision records your team inherits.
Build or buy decisions
For each component: buy a product, configure a platform, build on APIs, or self-host. Decided on cost, control and lock-in.
Model-agnostic design
A gateway and stable interfaces between your code and every model, so switching provider is configuration plus an eval run.
Data architecture for AI
Where each fact lives, how it reaches the model, who may see it and how changes flow through, with lineage you can audit.
Agent and workflow architecture
Which work is an agent, which is a fixed workflow and which is plain code, and how they hand off to people.
Security and tenancy design
Trust boundaries, data residency, tenant separation and least-privilege access designed in from the first diagram.
Cost architecture
Unit economics per request and per user, modelled before build, so the design fits the budget at full volume.
Reviews and decision records
Architecture reviews of pilots and vendor proposals, and decision records that explain every major choice to whoever comes next.
03 / The swap test
What has to change when something changes.
The test of an AI architecture is how much breaks when one part is replaced. Pick a change to see what it touches in our reference design.
Channels
WhatsAppWeb appEmailInternal console
Orchestration
AgentsDurable workflowsTool interfaces
Intelligence
AI gatewayModelsRetrieval
Data and systems
Vector indexPostgresERP connectorCRM connector
Operate
Evals and tracesSecurity and auditCost tracking
What changes
Gateway configuration and one eval run to confirm quality holds.
What stays
Agents, workflows, prompts, data stores and every integration.
Size of change
Small: configuration only
What changes
The retrieval adapter and a re-index job, then a retrieval eval.
What stays
Models, agents, channels and the documents themselves.
Size of change
Small: one adapter and a re-index
What changes
One new channel adapter beside the existing ones.
What stays
Agents, models, data and approvals; the new channel reuses all of them.
Size of change
Small: one new adapter
What changes
Models served on your GPUs behind the same gateway API; data stores redeployed; security controls re-verified.
What stays
Application code, agents, workflows and channels.
Size of change
Larger: hardware and security sign-off
What changes
The ERP connector behind the same tool interface.
What stays
Agents and prompts call the same tools; they never knew which ERP was behind them.
Size of change
Medium: paced by the ERP project
04 / Build or buy
Build vs buy for AI: a decision framework.
We decide per component, not per project. Most systems end up a mix: bought where the problem is generic, built where it is yours.
| Option | Best when | Effort to start | Control | Lock-in risk |
|---|---|---|---|---|
| Buy an AI product | The problem is generic and the product fits your process as it is | Lowest | Low | High |
| Configure a platform | Common pattern, but your rules and data need tailoring | Low to medium | Medium | Medium |
| Build on model APIs | The workflow is specific to you or a source of advantage | Medium to high | High | Low with a gateway |
| Build and self-host | Strict data rules, high volume or a need for full control | Highest | Full | Lowest |
Questions we answer before choosing
Is it core?
- Does this workflow set you apart?
- Would a rival buy the same tool?
Whose data?
- Can the data leave your network?
- Who owns outputs and logs?
Cost at volume
- Per-seat vs per-request at 10× use
- Exit cost if prices rise
Fit with systems
- APIs into your ERP and CRM?
- Works with your SSO and audit?
Who runs it
- Skills your team has today
- Support after handover
Way out
- Can you export data and prompts?
- What breaks if the vendor leaves?
05 / How it ships
Map, decide, prove, hand over.
Architecture you can trust is proven in code, not only drawn. The riskiest part of the design runs before the full build starts.
Designed with the team that will own itStep 01 · Map
Current state
Systems, data flows, existing AI tools, contracts and constraints mapped as they really are.
- Current-state diagram
- Constraints list
- Vendor inventory
Step 02 · Decide
Target design
Target architecture, build-or-buy calls and cost model, each recorded with options and reasons.
- Target architecture
- Decision records
- Friday review
Step 03 · Prove
Riskiest piece first
The part most likely to fail is built and tested on real data before the full build is committed.
- Working spike
- Swap test run
- Design updated
Step 04 · Hand over
Roadmap and standards
Build roadmap, reference architecture and standards handed to your team and the build squad.
- Roadmap
- Architecture standards
- Walkthrough session
06 / Reference stack
Labs 24 reference architecture.
The building blocks we start from. Every one has an alternative we can swap in when your constraints call for it.
Surfaces and orchestration
LangGraphTemporaln8nWhatsApp, SMS, emailInternal console
Models and retrieval
ClaudeGPTGeminiLlamaQwenMistralOllamavLLMQdrantPineconeLlamaIndexNeon
Systems of record and operations
ERPCRMTMSBillingLangfuseEval suitesAlerting
Security boundaries and compliance evidence for the architecture are covered in Security & Compliance.
07 / In production
AI architectures in production.
Under NDA, so clients stay unnamed. Each system runs on the client's own infrastructure with full IP transfer.
Digital banking offer engine
Designed as a self-contained module with clean interfaces, so it could be licensed and deployed inside a bank's stack.
Modular by design
Law practice document assistant
Architected for self-hosted models from day one, with the same interfaces a cloud model would use.
Self-hosted ready
Multi-app coupon campaign engine
One multi-tenant core serving several consumer apps, with each app added as configuration rather than a new build.
Multi-tenant core
08 / The squad around it
Architecture sets the shape the squad builds in.
Each discipline builds inside the layers and interfaces the architecture defines.
01
Agent Engineering
Agents inside the orchestration layer.
02LLM Engineering
Models behind the gateway.
03Data & Retrieval
The data layer and its lineage.
05Production & Cloud
Where each layer runs.
07Enterprise Solutions
Standards rolled out across teams.
11Security & Compliance
Trust boundaries in the design.
09 / FAQ
AI architecture FAQ.
What is AI architecture?
AI architecture is the design of how an AI system's parts fit together: channels, orchestration, models, retrieval, data, integrations, security and operations. Good AI architecture keeps each layer replaceable, so a new model, database or channel is a configuration change rather than a rebuild.
What does a model-agnostic AI architecture mean?
It means application code never talks to a specific model provider directly. Calls go through a gateway with a stable interface, prompts and evaluations are versioned separately, and switching from one model to another is a configuration change followed by an eval run.
How do you avoid AI vendor lock-in?
Put a gateway between your code and every model, keep data and prompts in stores you control, use open standards such as MCP for tools, prefer components you can export from, and record the exit path for every bought product before signing.
Should we build or buy AI?
Decide per component. Buy where the problem is generic and a product fits your process as is; configure a platform when you need tailoring; build on model APIs where the workflow is specific to you; self-host when data rules, volume or control demand it. Most systems end up a mix.
What is an AI reference architecture?
A layered template that every AI use case in a company follows: channels, orchestration, models and retrieval, data and systems of record, plus operations such as evals, security and cost tracking. It lets new teams reuse proven parts instead of starting from zero.
Agent, workflow or plain code: how do you choose?
Use plain code for fixed rules, a durable workflow for known multi-step processes, and an agent only where the steps genuinely depend on judgement. Labs 24 maps each step of the work and picks the simplest option that does the job reliably.
What is an architecture decision record?
A short document that records one significant design choice: the context, the options considered, the decision and its consequences. Labs 24 writes one for every major AI architecture choice so future teams know why the system is built the way it is.
What are the steps in an AI architecture engagement?
Labs 24 maps the current state, agrees the target architecture and build-or-buy decisions with written records, proves the riskiest piece in code on real data, then hands over the roadmap and standards to your team.
Can you review an existing AI pilot or vendor proposal?
Yes. Labs 24 reviews pilots and vendor proposals for lock-in, data handling, security, cost at scale and fit with your systems, and recommends what to keep, change or replace.