Here's the Exact Hardware You Need. Nothing More.
The AI steering committee reached the infrastructure question at 11:47am. The CTO asked the IT director what servers would be needed. The IT director mentioned GPU clusters. Someone referenced an article about NVIDIA's H100 chips. Someone else brought up the €2 million cloud migration estimate. The meeting ran over. The decision got deferred.
That conversation plays out in organizations every week. Not because the hardware question is hard. It gets answered with information about training AI models instead of running them — two different workloads with completely different hardware requirements.
This article is the answer that meeting needed.
---
Training Is Not Inference
Two activities look similar and require completely different infrastructure.
Training a language model — building it from scratch, teaching it on billions of words of text — requires thousands of GPUs running continuously for weeks. OpenAI trains GPT-4 on clusters that cost hundreds of millions of dollars. That's the infrastructure behind the headlines.
Running a deployed AI model — answering employee questions, analyzing documents, generating reports — is called inference. Inference is what your organization actually needs. And inference hardware is an entirely different category.
What that hardware actually looks like: compute equivalent to two to four high-end workstations for 500 to 5,000 users. Not a data center. Not exotic liquid cooling. Not a custom rack. A standard rack server with a single enterprise GPU fits in the rack you already have, runs on standard power, and is procured through the same channels as any other enterprise server.
AI hardware marketed to the enterprise — massive clusters, custom infrastructure, six-figure GPU configurations — is training hardware. Inference hardware gets far less press. That gap in coverage has delayed deployments that could have started 12 months ago.
You're not building a model. You're running one. The hardware for that is already in your data center.
---
What Each Sovereignty Level Needs
Three sovereignty levels. Three different hardware pictures.
SL1 — Hybrid Sovereign: Your sensitive data stays in your jurisdiction. AI processing for non-sensitive tasks uses existing cloud resources. If your organization already runs Azure, AWS, or a private cloud tenant, you have the foundation for SL1. No new hardware required to start.
SL2 — Data Sovereign: Nothing leaves your environment. This is where most regulated enterprise deployments land — healthcare, finance, legal, government. SL2 requires dedicated servers that don't share physical infrastructure with other organizations. Two standard enterprise servers handle it for most deployments up to 1,000 users. Hardware cost: €75,000 to €120,000.
SL3 — Full Air-Gap: Physically isolated. No external network connections. Every component, every model, every dependency lives on hardware inside your perimeter. This is for defense, classified facilities, and critical infrastructure. Hardware costs more and the specification is customized to the facility. Most commercial organizations don't need SL3 — every defense organization does.
---
The Reference Configuration for SL2
SL2 sovereign AI for 500 to 1,000 users requires two servers.
Your primary inference node is the server that actually runs the AI model and answers employee questions. The key component is the GPU: specifically, one NVIDIA L40S with 48 gigabytes of dedicated GPU memory (the memory that holds the AI model while it's running). That GPU can process approximately 1,500 words per second for an 8-billion-parameter model. At an average employee query of 300 words in and 150 words out, that's 3 to 4 seconds per response under 500 simultaneous users — fast enough for practical work. Full server spec: dual Intel Xeon 4th-generation processors, 512GB system RAM, and solid-state drives for the operating system and local cache.
Your storage and orchestration node holds your organization's documents, manages how queries get routed to the right models, and keeps the full audit log of every AI interaction. No GPU required here. Dual Xeon CPUs, 256GB RAM, and 40 terabytes of fast NVMe storage (the same type of high-speed solid-state storage used in enterprise database servers).
Both nodes connect via standard 10 gigabit networking — the same speed most enterprise data centers already use for database and application traffic.
Total hardware cost for this configuration: €75,000 to €120,000, depending on vendor and specific component selection. Available from Dell, HP, or Lenovo through standard enterprise procurement.
---
A Working Example
A financial services firm with 1,200 employees deployed SL2 sovereign AI on exactly this configuration — two Dell PowerEdge R750xa servers. One inference node with a single NVIDIA L40S GPU. One storage and orchestration node with high-memory CPU configuration.
Running the above configuration, the firm handles 300 concurrent AI sessions without queuing. It processes legal document reviews, client report drafting, compliance checking, and internal Q&A simultaneously. Total hardware investment: approximately €85,000.
Compare that to the firm's previous cloud AI spend: €12,000 per month, before any data sovereignty protections were in place. The hardware paid for itself in seven months while eliminating the compliance exposure entirely.
---
What About Model Quality?
Model quality comes up in the same meeting as the hardware question, and it's worth answering directly.
On most enterprise workloads — document analysis, summarization, question-answering on internal data, report generation, code assistance — Llama 3.1 70B and Mistral Large perform within 10 to 15 percent of GPT-4 on standard benchmarks. An employee drafting a client summary or the legal team reviewing a contract will not notice the difference.
Where you need the latest GPT-4 capability specifically, the SL1 hybrid architecture routes those queries to cloud AI while keeping all data processing sovereign. You get maximum capability where it's needed and maximum sovereignty everywhere else.
And the efficiency trend runs in one direction. The Llama 3.1 8-billion-parameter model runs in 8 gigabytes of GPU memory at production quality today. Two years ago, that same quality required 70 gigabytes. The hardware required for sovereign AI is shrinking every quarter as model efficiency improves.
---
The Timeline From Decision to Live
Procurement timelines and deployment timelines align well.
Standard enterprise GPU servers — the NVIDIA L4, L40S, and similar inference-grade hardware — are available on four to eight week lead times from standard distributors. The Leeloo Framework deploys in 8 to 12 weeks from contract signature. Start the hardware procurement the same week you sign the contract, and the servers arrive before deployment is complete.
Week one: hardware ordered.
Week five: hardware delivered.
Week six: racks installed and network configured.
Week twelve: sovereign AI system validated and live.
Energy consumption for this configuration runs approximately 1,000 to 1,500 watts — similar to two enterprise database servers. At EU electricity rates, that's €3,000 to €5,000 per year for 24-hour operation. A rounding error in most enterprise IT budgets.
---
"We Own Our AI"
Not every organization needs to say that. The ones that do know exactly what it means: AI that runs on your servers, trained on your data, logged on your storage, auditable on demand, under your physical and logical control at all times.
When a regulator asks where AI processing occurred — and they are starting to ask — the answer is your inventory list. Your own hardware, in your own facility. No cloud provider to request records from. No jurisdiction question to resolve. No subprocessor notification required.
Reaching that position requires hardware you own. The specification for that hardware is two servers, one GPU, and four to eight weeks of standard procurement.
Twelve weeks from today, your AI can run on your own infrastructure. The hardware question has an answer. The meeting where it stalled is the last time it needs to.
---
Leeloo is a sovereign AI implementation company based in Luxembourg. Implementation starts at €300K with deployments at SL1, SL2, or SL3. Book a 45-minute hardware specification call at [leeloo.ai](https://leeloo.ai) — we'll review your existing infrastructure and give you a procurement-ready spec for your specific user load.