AI Platforms & InfrastructureAI Infrastructure & LLMOps
Private AI Deployment Sprint
One open-weight model serving real production traffic from an OpenAI-compatible endpoint inside the client's own cloud account or data centre, with no prompt or output leaving their boundary. It comes with a benchmark report that states the throughput the endpoint sustains at agreed latency targets and its measured cost per million tokens, set against what the client pays today.
Private AI Deployment Sprint
$40K–$70K
- Duration: 4 weeks
- Tier: Build
- Format: Fixed scope
You leave withOpen-weight model in production in the client's VPC or on-premise, with benchmarked throughput and cost
Book a Free SessionSound familiar?
Legal won't let us send customer data to OpenAI, so the AI pilot is stuck in a sandbox with fake data. Someone on the team got Llama running on a GPU box and it works on their laptop, but nobody knows if it will hold up when 300 people use it at once, what it really costs per month, or who gets paged when it falls over. Meanwhile the API bill for the stuff that is allowed to go out keeps doubling every quarter.
What you leave with
Production inference endpoint
Format: Running service in client environment
How we know it’s acceptablePasses the acceptance test suite and meets agreed p95 TTFT/ITL at the agreed concurrency for a 30-minute sustained run.
Model decision record
Format: Markdown in client repo
How we know it’s acceptableSigned off by the client decision-maker at the week-1 gate.
Infrastructure-as-code
Format: Terraform + Helm in client repo
How we know it’s acceptableA client engineer recreates staging from an empty account using only the README, with us watching and not typing.
Evaluation harness and golden set
Format: Code + data in client repo
How we know it’s acceptableRe-runnable by the client's team in one command. It reproduces the scores in the decision record within tolerance.
Benchmark and cost report
Format: PDF + raw results + scripts
How we know it’s acceptableRe-running the benchmark script gives results within an agreed tolerance. The cost line reconciles with the cloud bill for the test period.
Dashboards and alerts
Format: In the client's Grafana / on-call tool
How we know it’s acceptableEach alert fired at least once during the game day and reached the right person.
Runbook
Format: Markdown in client repo
How we know it’s acceptableThe client engineer executed model rollback and node-failure recovery from it during game day, without our help.
Security notes
Format: Markdown
How we know it’s acceptableReviewed and accepted by the client infosec reviewer.
Handover recording
Format: Video
How we know it’s acceptableDelivered. Client confirms access.
Also included
- A gateway layer with SSO-linked API keys, per-team rate limits and usage metering, so the endpoint can be shared across teams without a free-for-all.
How it runs
4 weeks
Week −2 to 0
Intake
Client raises the GPU quota or capacity reservation. They create a dedicated account/subscription/project and give us federated, least-privilege, time-boxed access. Their domain expert collects 100-300 real examples. We run the intake gate.
OutSigned intake checklist. GPU nodes are actually launchable (we test-launch one). Kick-off date confirmed.
Week 1
Profile and decide
Traffic profile (input/output token distribution, concurrency, peak). SLOs agreed. Licence check on the shortlist. Up to three candidate models were evaluated on the golden set at BF16 and quantised. GPU sizing. Infosec architecture review held now, not in week 4.
OutModel decision record. Agreed SLOs and quality bar. Go/no-go decision signed by the client.
Week 2
Build the platform
Terraform for network, GPU node pool and storage. Weights pulled once into a private bucket and checksummed. Kubernetes with the NVIDIA GPU Operator. vLLM (SGLang if the workload is prefix-heavy). Gateway with SSO-linked keys and rate limits. Prometheus/Grafana/DCGM. Egress locked down.
OutWorking staging endpoint, reproducible from IaC. First smoke-test numbers.
Week 3
Benchmark and tune
GuideLLM sweeps against the real traffic shape. Tune batch size, max sequences, prefix caching and tensor parallelism. Re-check quality after each change. Autoscaling. Failure tests: pod kill, node loss, cold start.
OutBenchmark report v1: goodput at SLO, cost per million tokens, utilisation curve, failure-test results.
Week 4
Production and handover
Production deployment from the same IaC. Shadow or canary traffic. Alerts wired to their on-call. Runbook. Two game-day drills run by their engineers (model rollback, node failure). Final report. Paired handover.
OutProduction endpoint live. Final benchmark and cost report. Runbook. Recorded handover. Acceptance sign-off.
Is this right for you?
This is for you if
- Buyerthe person who owns the platform the AI workloads run on and the bill that comes with it. Usually that is a Head of Platform/Infrastructure, a VP Engineering or a CTO. In regulated firms it can be the person who owns the "we cannot send this data to a third-party model" problem.
- Also has to say yesInformation Security / CISO: they sign off the architecture, the network isolation and the model-weight provenance.
- Also has to say yesThe cloud account owner or FinOps: they approve the GPU quota request and the run-rate, which lands on the client's cloud bill, not ours.
- Also has to say yesLegal or procurement: they clear the model licence (for example the Llama Community Licence pass-through terms) and our access to their environment.
- Also has to say yesFinance, above the client's own approval threshold.
This is the wrong answer if
- Low or spiky volume with no residency mandate. Below roughly a few million tokens a day of sustained traffic, a managed API is cheaper once you count GPU idle time and the engineer who owns the server. So is an open-weight model served in-region by the cloud provider, such as Amazon Bedrock, Azure AI Foundry or Vertex AI. We say so on the first call and point them there.
- The model has to match a frontier closed model on their task, and the week-1 evaluation shows it does not. If no licence-cleared open model comes within the agreed margin on their own examples, self-hosting only moves the problem. We stop at the week-1 gate.
- Nobody on their side will own it after we leave. A self-hosted model is a service with on-call, CUDA/driver upgrades, security patches and model refreshes. Without a named owner and an on-call rota, it gets switched off within two quarters, or left unpatched. vLLM alone had several remote-code-execution advisories in 2025-26.
- They cannot secure the GPUs before day one. Quota is not capacity. A raised quota in a region that has no free H100/H200 capacity provisions nothing. We will not start a fixed-price clock on hardware we do not control.
- What they actually need is fine-tuning, RAG or an application. This sprint deploys and operates a model. It does not train one or build the feature on top. Selling it as "private ChatGPT for the company" sets the wrong expectation.
- Fully air-gapped or classified environments. Physical-media transfer of weights and images, plus restricted engineer access, does not fit four weeks. It needs a longer variant.
What’s included, and what isn’t
In scope
- one primary open-weight generative model (an embedding model can be co-deployed only if named at quote time)
- one staging and one production environment in one cloud region or one on-premise site
- a model shortlist and evaluation of up to three candidates
- serving engine set-up and tuning
- gateway and auth
- observability
- benchmarking
- runbook and handover
Out of scope
- GPU and cloud costs: these are billed to the client's account during and after the sprint and are not part of our fee.
- buying or racking hardware
- fine-tuning or training
- building the application, RAG pipeline or chatbot
- writing the golden set's correct answers (their domain expert owns that, we structure it)
- changes to their identity provider, network core or SIEM beyond a documented integration point
- NVIDIA AI Enterprise or other vendor licences
- a second model, region or environment
- production support after handover
Before we start
Before day one
The intake gate. The clock does not start until every line is ticked. This is also where the cloud onboarding happens.
- GPU capacity is real, not just quota. We have test-launched the target GPU node in the target region. Quota approval can take a day or more, and in constrained regions capacity may not exist even with quota. Where on-demand is unreliable, the client has bought reserved capacity (for example AWS EC2 Capacity Blocks, which can be booked up to eight weeks ahead) covering weeks 2-4 at least.
- A dedicated cloud account/subscription/project exists inside the client's organisation, with billing owned by them. BigStep engineers get access through the client's SSO (federated role, MFA, time-boxed, least privilege). We never use long-lived access keys or a BigStep-owned account.
- Network path is agreed: how our engineers reach the private environment (bastion, SSM/IAP, VPN), and how the client's applications will reach the endpoint (private link, peering, internal load balancer). The network team has named a contact.
- The model licence is cleared by their legal team for the shortlisted models (relevant for Llama-family terms in particular).
- A golden set of 100-300 real examples exists, with expected answers or a scoring rubric from their domain expert. A cleaned demo subset does not count.
- Named people: a platform engineer, an infosec reviewer, a domain expert, and a decision-maker who will answer within 24 hours.
- On-premise only: the GPU servers are racked, on supported drivers/OS, reachable, and have local storage sized for the weights (a 70B model at BF16 is about 140 GB before replicas).
What we’ll need from your team
- Cloud/platform engineer: about 6 hours a week in weeks −2 to 1 (accounts, quota, SSO roles, network), then about 3 hours a week. They also run the game-day drills in week 4, which takes about one more half-day.
- Infosec reviewer: about 3 hours in week 1 (architecture review) and about 4 hours in week 4 (sign-off). If they first see the design in week 4, week 4 slips.
- Domain expert: about 6 hours before kick-off to build the golden set, 3 hours in week 1 to judge model outputs, and 2 hours in week 3 to re-check quality after quantisation.
- Application owner: about 2 hours a week to provide traffic samples and plan the week-4 canary traffic.
- Decision-maker: available within 24 hours for the week-1 go/no-go and any scope question.
Pricing
What we need to give you a quote
- Current and 6-month projected request volume, plus a sample of 50-100 real (redacted) prompts with typical output length. This sets GPU count more than any other input.
- Target latency (or "it should feel like ChatGPT") and whether it is interactive or batch.
- Environment: which cloud and region, or on-premise hardware inventory (GPU model, count, driver/OS, Kubernetes or not).
- Residency and security constraints in writing: the data classification, any regulator (e.g. India's DPDP Act, sector regulators, GDPR, HIPAA for US clients), and whether outbound internet from the serving nodes is allowed at all.
- Whether GPU quota or capacity reservations are already in place, and who in their organisation can request them.
- Their preferred model or shortlist, and whether legal has reviewed the licence.
After we hand over
Who owns it after we leave
A named engineer on the client's platform team owns the endpoint, the runbook and the evaluation harness, and is on their on-call rota for it. We say plainly what it displaces. At steady state, expect roughly half a day a week for patching, upgrades and capacity checks. Expect a further 1-2 days per model refresh, which happens every few months if they keep current. Engine security advisories have to be acted on within days, not quarters. If no named owner exists by week 4, we record that in the final report as the top risk.
What we measure
- Cost per million tokens before (their API bill) and after (cloud bill ÷ tokens served)
- p95 time-to-first-token and inter-token latency at target concurrency
- GPU utilisation
- Quality pass-rate vs the incumbent model on the golden set
- Time to qualify and roll out a new model version
Agreed in week 1 and reported at handover. The client's own cloud bill is the 'after' figure we want, since a number they measured is worth more than ours.
What comes next
- An AI Feature Sprintthat builds the application or RAG use case on top of the new private endpoint. This is the most natural follow-on, because the endpoint has no users until something is built on it.
- Ongoing LLMOps managed service (patching, upgrades, model refreshes, capacity).
- A second model or environment (e.g. an embedding or vision model, a DR region), sold as a smaller follow-on sprint.
- Fine-tuning, if the evaluation showed a quality gap that domain data could close.
Questions buyers ask us
Often you shouldn't, and we'll tell you that on this call. Self-hosting wins in three cases: the data must not reach any third-party service, including the cloud provider's model service; your volume is high and steady enough that a reserved GPU beats per-token pricing; or you need a model, version or quantisation the platform doesn't offer and pinned for as long as you choose.
Getting the first token is an afternoon. What takes the time: sizing GPUs to your real traffic, proving latency under 300 concurrent users rather than one, checking whether FP8 costs you accuracy on your data, locking the network down so the box can't phone home, and an upgrade path for when the next CVE lands.
For some tasks that is still true, and it's a fair objection. That is why week 1 ends with a go/no-go on your own examples. If no licence-cleared model gets within the margin we agree up front, we stop there and you pay for one week, not four.
Yes, if your traffic is spiky and small. That is the main reason this is the wrong answer for some companies. We size for your measured peak, show the utilisation curve, and set autoscaling floors. If the break-even doesn't work at your volume, the sheet will show it before you sign.
Swapping the model is a runbook procedure, not a project. You leave with the evaluation harness, so your team qualifies the next version against the same golden set and rolls it out behind the gateway with a rollback.