When your AI gets something wrong, the problem might be the data behind it.

Data engineering for AI is the work of building the data infrastructure that feeds machine learning models, large language models and retrieval systems. Traditional data engineering mostly prepares tidy, structured data for reports and dashboards. Data engineering for AI also has to handle documents, emails, scans, audio and video, keep them searchable and up to date, and make sure the AI finds the right piece when it needs it. We start by auditing your data, then build the pipelines, document processing and search layers the audit calls for.

Unstructured data is information that doesn’t sit in rows and columns, such as PDFs, emails, scanned documents, images, audio and video. An AI can’t use it as it is because the text has to be read out of it first, then split into pieces and indexed so the right piece can be found. A scanned contract is a picture until something reads it. We read each type with the right method, using tools such as AWS Textract and Google Document AI, and index the pieces with permissions attached.

Often because it was given the wrong information. The right passage may not have been found, a document may be out of date, or a table may have lost its headings when it was read. The way to tell is to test the search behind the answer: for a set of real questions, check whether the right passage came back. If it did and the answer was still wrong, the model is the problem. If it didn’t, the data is. Send us the questions your AI gets wrong and we’ll look at the data first, which is quicker than fixing wrong answers one at a time.

A data contract is a written, versioned agreement that says what data an agent may read and write and what shape that data should take. Agents need one because they act on data rather than just display it. If a field is renamed upstream, a dashboard shows a wrong number that someone eventually notices, but an agent that doesn’t know the field changed carries on and takes a wrong action. A contract catches the change before the agent acts on it. We write these contracts for your agents and version them, so a breaking change upstream, such as a renamed field, is caught before the agent acts on it.

Your data is ready for AI when you know where it lives, you know how good it is, you know who is allowed to see what, and an AI can find the right piece of it when it’s asked. If you can’t say yes to every one of those, the data probably isn’t ready yet. We can check all of them for you in a written audit that gives you an inventory of your data, a quality score for each source, a ranked list of the gaps that stop an AI from using it, and a costed roadmap for closing them. If your data turns out to be in good enough shape, that’s the answer, and the problem is somewhere else.

You stop it by indexing each piece of a document with its source’s access rules attached, and checking those rules for the person asking before anything reaches the AI. It’s much harder to add permissions after the fact, because the AI may already have found pieces it was never meant to show. We built this for a property management company, using a Bedrock Knowledge Base over its document system, Outlook and property management software, so each answer reflects what the person asking is allowed to see. We build the same kind of permission-aware indexing for your documents.

If your AI works from a small set of clean documents, or from one system that already has good search, you may not. A platform’s built-in knowledge features can be enough at that size. Data work starts to pay when you have many sources, when permissions differ between people, when data changes often, or when answers are wrong and you can’t tell why. A short audit of your data will tell you which side you’re on.