Technology
A knowledge assistant that answers questions from your documents can be built in an afternoon. A knowledge assistant you dare to use in a governmental setting cannot. Below is where the difference sits.
01
Permissions
“Can someone from social services reach HR documents through this thing?”
No, and that is the first thing set up, not the last. Nor the source reference: no pointer to a document they may not open.
This is why most projects stall at a general knowledge base holding only public documents. The moment personnel files, objections, care files and executive papers have to enter the same system, the question changes from “can it answer” to “can anyone demonstrate that the separation holds”.
That is stricter than it sounds. The answer must contain no content from a document the user has no right to. The source reference must not point to it. And the system must not confirm that such a document exists either, because that is information in itself.
We have this working, separated per department and per corpus.
02
Provenance
Every answer points to the document and the passage it came from. Not for tidiness, but because a member of staff who decides something on the basis of that answer has to be able to show what the decision rests on.
03
Staleness
“What happens to a document we withdraw?”
It disappears from the assistant along with it, within the period you specify. Not at the next big reindexing. Ask this of every supplier you speak to. The answer is vaguer than you would like more often than not.
Regulations are withdrawn, policy is revised, versions pile up. An assistant that quotes the old version because it happens to match better does more damage than it solves. Document management is therefore not an afterthought but part of the design.
04
Silence
“And what if it simply makes something up?”
Then you will see it, because a source is always attached and it opens with one click. You will not get an answer without a source. That is not a guarantee that the model never goes wrong, it is the reason you notice before anyone acts on it.
The hardest behaviour to achieve is no answer at all. A language model is built to answer, and it will do so for questions whose answer is not in your documents. An assistant that says it does not know is worth more than one that is often wrong.
05
Fragmentation
Documents are rarely in one place. SharePoint, a network drive, email, a line-of-business application, each with its own structure, metadata and permissions. Bringing those together without losing the permissions is the bulk of the work, and the least visible part.
These are the five that stop projects in practice, and this is what we have worked on. The setup runs on Microsoft Azure. Exactly how we solved it is something we show in a conversation, not on a page.
More questions we already knowWant to know whether this works for your organisation?
An exploratory conversation of one hour usually gives a clear picture. Get in touch and we will schedule it.
besst.ai. Your documents answer questions. Only for those allowed to see them.