Know What Your AI Costs and Where It Runs

LLMOps is the work of keeping a large language model reliable once people are using it. That means versioning the models and prompts, releasing changes gradually, rolling back a bad one, watching quality, and tracking what each request costs. MLOps does the same job for models you train yourself. LLMOps adds the parts that are particular to language models, such as prompts, hosted model providers and answers that differ from one run to the next. Our LLMOps Platform Launch (6 to 10 weeks, $70K to $140K) sets this up. We version every model and prompt, release a change to a few users first, test it on your own cases, and roll back to the last good version if it gets worse.

There’s no single number. It depends on the model, how long each request and answer are, and whether you pay a provider per token or run your own GPUs. The answer for your system comes from measuring three things: the tokens going in and out of each request, the price per token or per GPU hour, and how many requests you handle. If you’d like us to do the measuring, send us last month’s spend and our free Estimate My Run Cost comes back within three working days. For the full picture, our Inference Cost Teardown (2 weeks, $18K to $30K) breaks your token and GPU spend down by feature, customer and request.

Self-hosting usually starts to pay when your volume is high and steady, when most requests are simple enough for a smaller model, or when your data can’t leave your network. At low or uneven volume, a hosted API is often cheaper, because you pay for a GPU whether or not it’s busy. The crossover depends on your own volume and response-time target, so it has to be worked out from your numbers. Our Inference Cost Teardown does that by pricing three setups at your volume: your current stack tuned, a different mix of models, and self-hosting. We don’t resell models, hardware or hosting, so if a hosted API turns out to be cheaper, we’ll tell you.

An open-weight model is one whose trained weights are published, so you can download it and run it on your own hardware, as you can with Llama, Qwen, DeepSeek and Mistral. It isn’t always open source in the strict sense, because some come with licence terms about how you can use them. Open weights are what let you fine-tune a model on your own data and keep it inside your network. Our Private AI Deployment Sprint (4 weeks, $40K to $70K) deploys and tunes one of these models in your own cloud account and measures its speed and cost against the hosted model you use today. We check the publisher’s licence with you before choosing the model.

Yes. An open-weight model can run on your own servers, in your private cloud or on a network with no outside connection. The trade-offs are that you run and update it yourselves, the choice of models is narrower than a hosted API gives you, and you need the GPU hardware. Our Sovereign AI Readiness Assessment (2 to 3 weeks, $22K to $40K) works out what can run where and writes the evidence your security team will ask for. If you need your own hardware, our GPU Cluster Design Sprint (3 weeks, $28K to $50K) sizes the servers.

The biggest levers are caching answers to questions that repeat, sending each request to the cheapest model that can do the job, tuning how models are served so the same hardware handles more requests, and switching to a second provider when the first one fails. Which one matters most depends on where your spend actually sits, so measure it by feature, customer and request before changing anything. Our Inference Cost Teardown (2 weeks, $18K to $30K) does that, and the tuning and routing work is scoped from its plan. We’ve built caching around the questions that actually repeat in two of our agent systems, one for a market intelligence company and one for an advertising agency.

It depends on whether you need a cost review or a full platform. A review is a small fixed-price project, while a full LLMOps platform is a multi-week build. Our Inference Cost Teardown is $18K to $30K over 2 weeks, a Sovereign AI Readiness Assessment is $22K to $40K over 2 to 3 weeks, a Private AI Deployment Sprint is $40K to $70K over 4 weeks, and an LLMOps Platform Launch is $70K to $140K over 6 to 10 weeks. If you’re not sure which one you need, the free run-cost estimate or the Teardown shows you.

If your AI usage is small, your bill is steady and low, and your data is allowed to go to a hosted provider, you probably don’t. A hosted API is simple, and for many teams it stays the right choice. It’s worth looking again when the bill becomes a line someone asks about, when a compliance team says data can’t leave, or when a change at one provider would change your product. If you’re not sure which side you’re on, an Inference Cost Teardown will tell you. If your current setup is already the cheapest sensible one, that’s the answer.