AI That Ships Inside Your Product

For a lot of tasks, you should. The line is whether the AI needs to work over your data, inside your product, for your customers. A general assistant can’t see your documents, doesn’t know your schema and has no way to respect who’s allowed to see what. A vendor’s built-in AI can do all three, but only inside that vendor’s own boundary, and your customers’ questions often cross three systems at once. The parts that make an AI feature yours are the parts a general tool can’t do and a single vendor can’t reach. If you’re not sure which side of that line your idea falls on, BigStep’s free 90-minute session with a senior AI engineer will tell you, including when the answer is to use something off the shelf.

RAG is a way of having an AI feature look up the relevant passages from your own documents or data first, then hand them to the model along with the question, so the answer is based on your information instead of only what the model learned in training. An AI feature needs it whenever the answer depends on content the model has never seen, like your contracts, product documentation or customer records. The quality of the answers then depends less on the model and more on how retrieval is designed: how documents are split up, whether search combines keyword and semantic matching, how results are ranked and whether each answer cites its source passage. BigStep treats this as core engineering in every AI feature it builds, so context engineering and RAG architecture are included in each engagement and aren’t sold separately.

Prompt injection is when someone hides instructions inside content an AI feature reads, such as a document or an email, to make it misbehave. It matters most in the systems most worth building: an AI that reads your customers’ documents will eventually read one written to manipulate it. The defences are structural, not textual. Retrieved content is kept separate from instructions. The model’s available actions are limited to what the feature needs. Permissions are enforced at retrieval, so an injected instruction can’t reach data the user couldn’t open anyway. And any output that triggers an action is validated, not trusted. BigStep tests every AI system against this before it reaches production.

Running cost comes down to the cost of each call multiplied by how often the feature gets used, which is why a pilot that looked affordable can turn into a reason to limit usage once real volume arrives. The levers are prompt caching, context budgeting, controlling output length, semantic caching of repeated queries, and routing so cheap models handle easy requests and frontier models handle hard ones. Where volume justifies it, distilling a frontier model’s behaviour into a smaller open-weight model can cut cost several-fold on a well-scoped task. BigStep models the cost per call at your projected volume before anything is built and treats it as a design limit from the first week, so you have a cost-per-call figure before you have an invoice.

The model will change, and sooner than your roadmap assumes. Providers retire versions and update behaviour, sometimes without a version bump. The protection is the evaluation set built during the engagement: when a model changes, you re-run it and see what moved, instead of hearing about it from a customer. Nothing we build hard-codes one provider, and the model choice gets revisited, not assumed. It’s the main reason we build an evaluation set even on a three-week sprint.

Usually not. A prototype that retrieves the right answer most of the time has already solved the fun part. What’s left is the part that isn’t: a permission model built into retrieval and not bolted on after it, an evaluation set so a provider’s change is caught before a customer reports it, a cost per call that holds at real volume, and monitoring. That’s the Production Hardening Sprint, three to four weeks and $25K to $45K. If the prototype genuinely can’t be hardened, we’ll say so on the free review, not three weeks into a fixed-price engagement, and the honest answer then is an AI Feature Sprint instead.

Usually not at the start. Retrieval, prompting and a test set tend to come first, and fine-tuning is worth considering once real usage shows where it would help: a narrow task you run at high volume, a domain the base model handles poorly, or a cost you want to bring down by moving to a smaller model. The usual routes are LoRA and QLoRA on open-weight models, or distilling a larger model into a smaller one. BigStep treats model customisation and cost engineering as typically a second engagement, scoped once there’s enough usage to show where the work will have the most impact.