Why a general AI chatbot fails an audit
The question is not whether the AI was right. It is whether you can show what it was asked, what it read, and who asked it — eighteen months later, to someone who was not there.
August 2, 2026 · 6 min read
The question that arrives late
AI adoption in commercial real estate usually starts as a productivity story. Someone tries an assistant on a lease, it works, word spreads, and within a quarter several people are using it for real work.
The governance question arrives afterward, and rarely from inside the company. It comes from an auditor reviewing a shared-cost reconciliation, a lender doing diligence, an LP asking about process, or counsel responding to a tenant dispute. The question is some version of: how was this figure produced, and can you show me?
What is actually being asked
It helps to be precise, because "can we use AI" is not the question and never was. The questions that get asked are narrow and factual.
- Which documents informed this number, and can you produce them?
- Who ran the analysis, and were they authorized to see those documents?
- Was any confidential material sent to a third party, and under what terms?
- If we asked the same question today, would we get the same answer?
- Can you show us the trail, or are you asking us to take your word for it?
Notice that none of these is about whether the model is smart. Model quality is not the audit surface. Provenance is.
Why a chat history is not a record
The instinctive answer is to point at the conversation. That fails for reasons that have nothing to do with AI and everything to do with what a record is.
A chat history lives in an individual's account, which means it can be deleted by that individual, disappears when they leave, and is not addressable by anyone else. It records what was displayed, not what was retrieved — the model may have read forty documents and cited two. It has no notion of authorization, so it cannot demonstrate that the person asking was permitted to see what they saw. And it is mutable in ways an append-only log is not.
For an auditor, "here is a screenshot of a conversation" is roughly as persuasive as "here is a photo of my notes." It is evidence that something happened. It is not evidence of what.
What a defensible record looks like
The bar is not exotic. It is the same bar every other system of record in your organization already meets.
- Append-only: entries are written and never edited or deleted, so the log cannot be tidied after the fact.
- Attributed: every entry ties to a specific user and a timestamp.
- Complete on retrieval: it records what was searched and what was returned, not only what was shown.
- Correlated: a single identifier links the request, the retrieval, and the response, so a question can be reconstructed end to end.
- Scoped: it demonstrates that the query ran inside an authorization boundary, rather than assuming it.
The uncomfortable middle period
Most organizations we talk to are in a specific and awkward position: AI is already being used on confidential documents, informally, by good people trying to do their jobs faster, on accounts the company does not control.
There is no version of this where the answer is to send a memo banning it. The tool is too useful and the ban is unenforceable. The workable answer is to give people something at least as good inside a boundary — sanctioned access, roles the company sets, and a record that exists whether or not anyone thinks to keep one.
The strongest argument for purpose-built software is not that it answers better. Often it does, on domain documents. The argument is that eighteen months from now, when someone asks how a number was produced, there is something to show them.
See it on your own documents
The fastest way to settle any of this is to run it against a lease you already know the answers to.