AI Platforms & InfrastructureAI Infrastructure & LLMOps
Private AI Deployment Sprint
One open-weight model serving real production traffic from an OpenAI-compatible endpoint inside your own cloud account or data centre, with no prompt or output leaving your boundary. It comes with a benchmark report that states the throughput the endpoint sustains at agreed latency targets and its measured cost per million tokens, set against what you pay today.
Private AI Deployment Sprint
$40K–$70K
- Duration: 4 weeks
- Tier: Build
- Format: Fixed scope
You leave withOpen-weight model in production in the client's VPC or on-premise, with benchmarked throughput and cost
Book a Free SessionSound familiar?
Legal won't let us send customer data to OpenAI, so the AI pilot is stuck in a sandbox with fake data. Someone on the team got Llama running on a GPU box and it works on their laptop, but nobody knows if it will hold up when 300 people use it at once, what it really costs per month, or who gets paged when it falls over. Meanwhile the API bill for the stuff that is allowed to go out keeps doubling every quarter.
What you leave with
Production inference endpoint
Format: Running service in your environment
How we know it’s acceptablePasses the acceptance test suite and meets agreed p95 TTFT/ITL at the agreed concurrency for a 30-minute sustained run.
Model decision record
Format: Markdown in your repo
How we know it’s acceptableSigned off by your decision-maker at the week-1 gate.
Infrastructure-as-code
Format: Terraform + Helm in your repo
How we know it’s acceptableYour engineer recreates staging from an empty account using only the README, with us watching and not typing.
Evaluation harness and golden set
Format: Code + data in your repo
How we know it’s acceptableRe-runnable by your team in one command. It reproduces the scores in the decision record within tolerance.
Benchmark and cost report
Format: PDF + raw results + scripts
How we know it’s acceptableRe-running the benchmark script gives results within an agreed tolerance. The cost line reconciles with the cloud bill for the test period.
Dashboards and alerts
Format: In your Grafana / on-call tool
How we know it’s acceptableEach alert fired at least once during the game day and reached the right person.
Runbook
Format: Markdown in your repo
How we know it’s acceptableYour engineer executed model rollback and node-failure recovery from it during game day, without our help.
Security notes
Format: Markdown
How we know it’s acceptableReviewed and accepted by your infosec reviewer.
Handover recording
Format: Video
How we know it’s acceptableDelivered. Client confirms access.
Also included
- A gateway layer with SSO-linked API keys, per-team rate limits and usage metering, so the endpoint can be shared across teams without a free-for-all.
How it runs
4 weeks
Week −2 to 0
Intake
You raise the GPU quota or capacity reservation. You create a dedicated account/subscription/project and give us federated, least-privilege, time-boxed access. Your domain expert collects 100-300 real examples. We run the intake gate.
OutSigned intake checklist. GPU nodes are actually launchable (we test-launch one). Kick-off date confirmed.
Week 1
Profile and decide
Traffic profile (input/output token distribution, concurrency, peak). SLOs agreed. Licence check on the shortlist. Up to three candidate models were evaluated on the golden set at BF16 and quantised. GPU sizing. Infosec architecture review held now, not in week 4.
OutModel decision record. Agreed SLOs and quality bar. Go/no-go decision signed by you.
Week 2
Build the platform
Terraform for network, GPU node pool and storage. Weights pulled once into a private bucket and checksummed. Kubernetes with the NVIDIA GPU Operator. vLLM (SGLang if the workload is prefix-heavy). Gateway with SSO-linked keys and rate limits. Prometheus/Grafana/DCGM. Egress locked down.
OutWorking staging endpoint, reproducible from IaC. First smoke-test numbers.
Week 3
Benchmark and tune
GuideLLM sweeps against the real traffic shape. Tune batch size, max sequences, prefix caching and tensor parallelism. Re-check quality after each change. Autoscaling. Failure tests: pod kill, node loss, cold start.
OutBenchmark report v1: goodput at SLO, cost per million tokens, utilisation curve, failure-test results.
Week 4
Production and handover
Production deployment from the same IaC. Shadow or canary traffic. Alerts wired to your on-call. Runbook. Two game-day drills run by your engineers (model rollback, node failure). Final report. Paired handover.
OutProduction endpoint live. Final benchmark and cost report. Runbook. Recorded handover. Acceptance sign-off.
Is this right for you?
This is for you if
- Buyerthe person who owns the platform the AI workloads run on and the bill that comes with it. Usually that is a Head of Platform/Infrastructure, a VP Engineering or a CTO. In regulated firms it can be the person who owns the "we cannot send this data to a third-party model" problem.
- Also has to say yesInformation Security / CISO: they sign off the architecture, the network isolation and the model-weight provenance.
- Also has to say yesThe cloud account owner or FinOps: they approve the GPU quota request and the run-rate, which lands on your cloud bill, not ours.
- Also has to say yesLegal or procurement: they clear the model licence (for example the Llama Community Licence pass-through terms) and our access to their environment.
- Also has to say yesFinance, above your own approval threshold.
When this isn't the right fit
- Your volume is low or spiky, and no residency rule applies. Below a few million tokens a day of sustained traffic, a managed API, or an open-weight model served in-region by your cloud provider, is cheaper once you count GPU idle time and the engineer who owns the server. We say so on the first call and point you there.
- The model has to match a frontier closed model on your task, and the week-1 evaluation shows it does not. If no licence-cleared open model comes within the agreed margin on your own examples, self-hosting only moves the problem. We stop at the week-1 gate.
- Nobody on your side will own it after we leave. A self-hosted model is a service with on-call, driver upgrades, security patches and model refreshes. Without a named owner and an on-call rota, it gets switched off or left unpatched.
- You cannot secure the GPUs before day one. Quota is not capacity. We will not start a fixed-price clock on hardware we do not control.
- What you need is fine-tuning, RAG or an application. This sprint deploys and operates a model. It does not train one or build the feature on top.
- Your environment is fully air-gapped or classified. Moving weights and images by physical media does not fit four weeks. It needs a longer variant.
What’s included, and what isn’t
In scope
- one primary open-weight generative model (an embedding model can be co-deployed only if named at quote time)
- one staging and one production environment in one cloud region
- a model shortlist and evaluation of up to three candidates
- serving engine set-up and tuning
- gateway and auth
- observability
- benchmarking
- runbook and handover
Out of scope
- GPU and cloud costs: these are billed to your account during and after the sprint and are not part of our fee.
- buying or racking hardware
- fine-tuning or training
- building the application, RAG pipeline or chatbot
- writing the golden set's correct answers (your domain expert owns that, we structure it)
- changes to your identity provider, network core or SIEM beyond a documented integration point
- NVIDIA AI Enterprise or other vendor licences
- a second model, region or environment
- production support after handover
Before we start
Before day one
The intake gate. The clock does not start until every line is ticked. This is also where the cloud onboarding happens.
- GPU capacity is real, not just quota. We have test-launched the target GPU node in the target region. Quota approval can take a day or more, and in constrained regions capacity may not exist even with quota. Where on-demand is unreliable, you have bought reserved capacity (for example AWS EC2 Capacity Blocks, which can be booked up to eight weeks ahead) covering weeks 2-4 at least.
- A dedicated cloud account/subscription/project exists inside your organisation, with billing owned by you. BigStep engineers get access through your SSO (federated role, MFA, time-boxed, least privilege). We never use long-lived access keys or a BigStep-owned account.
- Network path is agreed: how our engineers reach the private environment (bastion, SSM/IAP, VPN), and how your applications will reach the endpoint (private link, peering, internal load balancer). The network team has named a contact.
- The model licence is cleared by your legal team for the shortlisted models (relevant for Llama-family terms in particular).
- A golden set of 100-300 real examples exists, with expected answers or a scoring rubric from your domain expert. A cleaned demo subset does not count.
- Named people: a platform engineer, an infosec reviewer, a domain expert, and a decision-maker who will answer within 24 hours.
- On-premise only: the GPU servers are racked, on supported drivers/OS, reachable, and have local storage sized for the weights (a 70B model at BF16 is about 140 GB before replicas).
What we’ll need from your team
- Cloud/platform engineer: about 6 hours a week in weeks −2 to 1 (accounts, quota, SSO roles, network), then about 3 hours a week. They also run the game-day drills in week 4, which takes about one more half-day.
- Infosec reviewer: about 3 hours in week 1 (architecture review) and about 4 hours in week 4 (sign-off). If they first see the design in week 4, week 4 slips.
- Domain expert: about 6 hours before kick-off to build the golden set, 3 hours in week 1 to judge model outputs, and 2 hours in week 3 to re-check quality after quantisation.
- Application owner: about 2 hours a week to provide traffic samples and plan the week-4 canary traffic.
- Decision-maker: available within 24 hours for the week-1 go/no-go and any scope question.
Pricing
What we need to give you a quote
- Current and 6-month projected request volume, plus a sample of 50-100 real (redacted) prompts with typical output length. This sets GPU count more than any other input.
- Target latency (or "it should feel like ChatGPT") and whether it is interactive or batch.
- Environment: which cloud and region.
- Residency and security constraints in writing: the data classification, any regulator (e.g. HIPAA and state privacy laws for US clients, GDPR, sector regulators), and whether outbound internet from the serving nodes is allowed at all.
- Whether GPU quota or capacity reservations are already in place, and who in your organisation can request them.
- Your preferred model or shortlist, and whether legal has reviewed the licence.
After we hand over
Who owns it after we leave
A named engineer on your platform team owns the endpoint, the runbook and the evaluation harness, and is on your on-call rota for it. We say plainly what it displaces. At steady state, expect roughly half a day a week for patching, upgrades and capacity checks. Expect a further 1-2 days per model refresh, which happens every few months if you keep current. Engine security advisories have to be acted on within days, not quarters. If no named owner exists by week 4, we record that in the final report as the top risk.
How success is measured
- Cost per million tokens before (your API bill) and after (cloud bill ÷ tokens served)
- p95 time-to-first-token and inter-token latency at target concurrency
- GPU utilisation
- Quality pass-rate vs the incumbent model on the golden set
- Time to qualify and roll out a new model version
Agreed in week 1 and reported at handover. Your own cloud bill is the after figure.
What comes next
- 3 weeks · $20K–$35KAn AI Feature Sprintthat builds the application or RAG use case on top of the new private endpoint. This is the most natural follow-on, because the endpoint has no users until something is built on it.
- Ongoing LLMOps managed service (patching, upgrades, model refreshes, capacity).
- A second model or environment (e.g. an embedding or vision model, a DR region), offered as a smaller follow-on sprint.
- Fine-tuning, if the evaluation showed a quality gap that domain data could close.
Questions buyers ask us
Often you shouldn't, and we'll tell you that on this call. Self-hosting wins in three cases: the data must not reach any third-party service, including the cloud provider's model service; your volume is high and steady enough that a reserved GPU beats per-token pricing; or you need a model, version or quantisation the platform doesn't offer and pinned for as long as you choose.
Getting the first token is an afternoon. What takes the time: sizing GPUs to your real traffic, proving latency under 300 concurrent users rather than one, checking whether FP8 costs you accuracy on your data, locking the network down so the box can't phone home, and an upgrade path for when the next CVE lands.
For some tasks that is still true, and it's a fair objection. That is why week 1 ends with a go/no-go on your own examples. If no licence-cleared model gets within the margin we agree up front, we stop there and you pay for one week, not four.
Yes, if your traffic is spiky and small. That is the main reason this is the wrong answer for some companies. We size for your measured peak, show the utilisation curve, and set autoscaling floors. If the break-even doesn't work at your volume, the sheet will show it before you sign.
Swapping the model is a runbook procedure, not a project. You leave with the evaluation harness, so your team qualifies the next version against the same golden set and rolls it out behind the gateway with a rollback.