The short version
The data was never the problem
A procurement platform: eight modules, years of history, no data quality issues. Contracts, variations, options, invoicing instructions, deposits. All recorded properly.
Staff still could not get answers out of it — not because the answers were missing. Reaching them meant knowing which of the eight registers held the record, which filter narrowed it, and how the vendor’s name was spelled three years ago.
The interface was fine. The interface was also the problem. Every interface of that kind quietly assumes the person already knows where to look.
A misspelling should not be a dead end
Search a vendor register for a name typed slightly wrong and you get nothing. No suggestion, no near match. An empty table and a record count of zero.
A person watching over your shoulder would say you mean Blue Harbour and move on. Search cannot, because it matches strings. We wanted it matching the entities that actually exist: resolve the name to a real vendor first, then answer the question.
The card underneath the sentence is the part that matters. An assistant that returns prose alone asks to be trusted. One that returns the record too can be checked in two seconds.
Some questions do not belong to any one screen
A manager asking what one of their officers has in flight is asking across four registers at once. There is no screen for that, and there never was going to be.
Look at the bottom of that answer. Three registers were answered. The fourth was not, and it says so.
Why the deposits register could not be answered
That register holds only the contract reference, deposit type, amount, date and status against each deposit. It records nobody as having raised or managed it.
There is therefore no supported way to filter it by person. The assistant states exactly that, and offers the query it can run instead. Getting that behaviour was harder than getting the first three sections right.
The refusal was the hard part
Every language model will invent a fluent, confident answer if you let it. In a chat window that is an annoyance. On a system whose numbers drive purchasing decisions it is disqualifying. One fabricated contract value destroys every correct answer the assistant ever gave.
An assistant that is right ninety-five per cent of the time, and untrustworthy about which five per cent, is worth less than no assistant at all.
The rule we designed firstSo we designed the refusal first and the answering second. The rule is narrow and boring: answer only from records retrieved on this request, and when they do not support the question, say what is held instead.
That screen is the assistant asked for contracts ending next month. The register has no end date field. Rather than infer one from submission dates, which would look right and be wrong, it names the fields it has.
In testing, this was the response that changed minds. Sceptical users started trusting the answers because they had seen it decline one.
Every refusal is a free gap analysis
A side effect we did not plan: the log of questions it could not answer turned out to be more useful than the log of questions it could.
That list is a record of the reports the business wanted and nobody built — in the words of the people who wanted them, at the moment they wanted them. Contract end dates were on it.
How we use that log now
Several things on it had never been raised as a requirement, because raising it meant a change request and a wait, and it was quicker to keep doing it by hand.
We now read it monthly and count the repeats. The top of the list tells you what to build next as evidence rather than opinion, and it costs nothing to collect because the system already generates it.
Which model answers is a setting
The question we are asked most is which model we used. It is the least durable decision in the build. Models get better, cheaper and deprecated far faster than a procurement system lives, so hard-wiring one in is a decision you will have to unpick.
Treating it as configuration keeps the trade-off between accuracy, cost and confidentiality with the organisation.
Why this keeps a self-hosted model realistic
Every request in this build goes through a single configurable provider. Pointing it at a model running on your own hardware is that setting plus an adapter, not a rewrite of the application.
In regulated and government-adjacent work, that property tends to matter more than any benchmark score.
What we deliberately did not change
No new tables. No migration. No change to approval routing or permissions. The assistant answers within what the signed-in user may already see, because the alternative is a system that quietly becomes a way to read other people’s records.
Why we set that constraint on day one
An assistant that needs schema changes has to be justified against every other demand on the same engineering budget, and it will lose.
One that reads what is already there can be switched on for a single team, evaluated honestly, and switched off without leaving a mark.
One team, real questions, a fixed period, and one number: how many calls to the department that owns the system it removes. Everything else is interesting. That one is decisive.
About these screenshots
The screenshots come from a working development build with seeded test records, not a live client system. The product branding and the signed-in account have been replaced.