Private LLM deployment & operations

Your own LLM. Your data stays yours. No per-token bills.

DigitalCare deploys open-source LLMs (Qwen, Llama, DeepSeek, gpt-oss, Mistral) as production infrastructure: Kubernetes, GPU nodes, vLLM inference, an OpenAI-compatible API. On dedicated EU GPU servers with a flat monthly bill, or inside your own environment where data never leaves your perimeter. We operate the whole stack 24/7.

Get a cost assessment Run the numbers

LLM inference in production for our own productsKubernetes · vLLM · llm-d · PrometheusInfrastructure work since 2012

Aurora of dotted data streams in sunset colors over a dark horizon

The one-line migration

Your code doesn’t change. Your data path does.

The endpoint is OpenAI-compatible. Point base_url at your private cluster and every prompt, document, and completion stays on hardware you control. Same SDKs, same tooling, same agents.

app.py · the only diff

client = OpenAI(
-   base_url="https://api.openai.com/v1",+   base_url="https://llm.yourcompany.internal/v1",    api_key=os.environ["LLM_API_KEY"],
)
r = client.chat.completions.create(
    model="qwen3-32b",  # your model, your GPUs
    messages=[...],
)

The economics, not the adjectives

Slide to your daily token volume. The number makes the argument.

Tokens processed per day

57M tokens/day

1M / day500M / day

Public API, est. / month

$17,039

GPT-4-class blended input/output rate. Scales with every token, forever.

Dedicated GPUs + operations, est. / month

$8,500

1 dedicated GPU node + 24/7 operations. Flat: the bill doesn’t follow your traffic.

Self-hosting saves ≈ $8,539/mo at this volume (indicative).

indicative breakeven ≈ 28M tokens/day

All figures are indicative estimates for orientation only, confirmed against your real workload, model choice, and environment during a free assessment. Not an offer or a price list. Assumptions: GPT-4-class API at a blended $10 per 1M tokens; per-node capacity of a 30B-class model on one H100 at realistic utilization (larger models fit fewer tokens per node); the base covers GPU rental plus managed operations.

Honest disqualifier: at low volumes a public API is cheaper and simpler, and we will tell you so. Self-hosting starts winning at sustained multi-million-tokens-per-day workloads: high-volume classification, extraction, internal copilots, agents.

One stack, two homes

Where it runs is one decision. We make it with you at assessment.

We run your model on dedicated EU GPU servers. You get a private OpenAI-compatible endpoint, API keys and quotas for your teams, and a flat monthly bill. Prompts and outputs never train anyone’s model and never leave the dedicated environment.

dedicated EU GPU serversprivate endpoint + keysflat monthly bill

Model catalog

The menu changes as open models improve. The infrastructure underneath doesn’t.

Model	Hardware it runs on	What it is good at
Qwen3-32B	1× H100 80GB	Strong multilingual generalist; tool use and agents.
gpt-oss-120B	1× H100 80GB	GPT-4-class reasoning on a single card; best value per GPU.
gpt-oss-20B	1× L40S 48GB	Light copilots, extraction, high-throughput classification.
Llama 3.3 70B	2× H100 80GB	Proven generalist with the largest tooling ecosystem.
Qwen3-235B-A22B	4× H100 80GB	Frontier-class MoE for the hardest reasoning workloads.
DeepSeek-V3 / R1	8× H100 80GB	Heavy reasoning and code; multi-node serving via llm-d.
Mistral Small 3.2 24B	1× L40S 48GB	Fast, efficient, EU-origin; latency-sensitive services.

A new model release is a rollout, not a re-platforming. See operations below.

The serving stack

Written for engineers. No adjectives.

kubernetes + gpu-operator

GPU nodes managed as cattle: NVIDIA GPU Operator handles drivers, device plugins, and node lifecycle; the cluster is declared in code and rebuildable.

vllm

The inference engine. Continuous batching and paged attention for production throughput; quantized and full-precision serving per model.

llm-d

CNCF Kubernetes-native distributed inference stack (co-developed by Red Hat, Google, and NVIDIA) for HA and multi-node serving of large models.

openai-compatible gateway

One endpoint, per-team API keys, quotas, and usage metering. Routes by model name; several models can share one cluster.

prometheus + grafana

Inference metrics that matter: time-to-first-token, tokens/sec, GPU utilization, queue depth, error rates, with alerting wired to on-call.

canary rollouts

Model updates ship as canaries behind the same endpoint: new weights take a traffic slice, get validated against live metrics, then promote or roll back.

The same AI operations layer that powers DigitalCare’s Metatron watches this stack: assistance supervised by senior engineers, not autonomy. We run LLM inference in production for our own products on this exact architecture.

How it works

Assessment is free. Operations are a subscription. Infrastructure is at cost.

Assess · free

Workload profiling, model selection against your actual tasks, GPU sizing, and a TCO comparison against your current API bill, including the honest answer if the API should stay.

Deploy

Infrastructure as code: cluster, GPU nodes, inference stack, OpenAI-compatible gateway, monitoring, and key management, in our hosting or your environment.

Operate · 24/7

Monitored fleet around the clock, model and stack upgrades as new open models land, capacity and cost reviews, incident response by senior engineers. Monthly operations subscription plus infrastructure at cost.

Questions engineers ask8 questionsCloseOpen

Which models can you host?

Any production-grade open-weights model: Qwen3 family, gpt-oss-20B and 120B, Llama 3.3, DeepSeek, Mistral, and others. Model selection is part of the assessment: the menu changes as open models improve; the infrastructure underneath doesn’t.

Your hardware or ours?

Either. Hosted by DigitalCare runs on dedicated EU GPU servers we operate. On-your-infrastructure deploys the same stack into your cloud or data center, so data never leaves your perimeter. It’s one decision we make together at assessment.

How does the cost comparison work, and when does the API stay cheaper?

We model your real token volume against dedicated GPU capacity plus operations. At low volumes a public API is cheaper and simpler, and we tell prospects when that is the case. Self-hosting starts winning at sustained multi-million-tokens-per-day workloads.

Is quality comparable to GPT-4-class APIs?

For many workloads yes, with the right model: classification, extraction, internal copilots and agents often match API quality once tuned. We validate against your actual tasks during assessment rather than promising it upfront.

Where does our data live?

Hosted deployments run on dedicated EU servers; prompts and outputs are never used for training and never shared. On-your-infrastructure deployments keep all data inside your perimeter.

What happens when a better open model comes out?

We swap it. That is part of operations: model upgrades run as canary rollouts behind the same OpenAI-compatible endpoint, so your code doesn’t change.

Can we run multiple models on one cluster?

Yes. The gateway routes by model name, so one GPU cluster can serve several models with per-model quotas and metering.

How long until production?

Typically weeks, not months. Confirmed at assessment once workload, model, and environment are fixed.

Show us your API bill.

We’ll model self-hosted economics for your real workload and tell you honestly which side of the breakeven you’re on, even when the answer is “keep the API.”

Get a cost assessment welcome@digitalcares.com

Anonymized case

LLM costs cut 55%, all prompts in-perimeter

→

From the blog

When self-hosting an LLM pays off

→