Building trust, an LLM thought experiment
You wake up one morning to a piece of code written in a language that was just invented by an agent. You're an explorer who needs to go there, understand the language, then the code, and maintain it until you retire (OK, we'll either be living on universal income or dead by then, but please give it a try).
Since it was invented by the model, there's no documentation or man page you can quickly grep through.
This hypothetical scenario is just a thought exercise to illustrate a question I've been asking myself since Claude Code launched: how do you build trust in an agent?
"Trust" might be a strong word, and yet I think it's the notion that best captures the human–agent relationship: trust is giving up control to someone else in exchange for a potentially better outcome. It's probabilistic in essence (you don't "trust" the Pythagorean theorem; it's irrefutable. The theorem doesn't want your trust, as it's the expression of a universal truth that doesn't require your presence to remain valid), and that's what makes it risky: there's an imbalance between the truster and the trusted, the truster carrying most of the risk as the price to pay for delegating ownership of the problem.
How do you build trust? I see trust as the result of an accumulation of concordant judgments, derived from a structured inference (let's say Bayesian) of facts. The accumulation matters here, practically speaking, because trust is built, not given: to give control away, you want to build as much confidence as possible (one could even say, from a statistical point of view) that the outcome outweighs the risk. And to do this, you need data, as much as you can gather, and lots of experiments too, to fuel your inner inference engine.
Trust isn't atomic: though it's directed at a delimited, contained entity (a coworker, a government, an institution), trust doesn't offer itself universally (something else can, and we'll see later why that's different). That's because trust sides with action: it's an implicit contract exercised on a task. Both parties are bound to the result, even if one carries more of the burden of failure. The keyword is task, not subject: I trust my daughter to make a good vinaigrette by herself. I don't trust her to pick the right ingredients in the first place. Same subject, different task, different level of trust. When I say trust sides with action, that's because it's grammatically implied: it's a verb. "I trust you" clearly expresses the delegation the subject makes towards the object. It's even clearer in French: "Je te fais confiance."
But there's another version you often hear too: "J'ai confiance en toi" (I can't find an idiomatic translation; "I have faith in you" is probably the closest). And that's not what we're after with our tentative definition. Having faith in someone is a form of laisser-faire, not a partial giving up. You are willingly abandoning judgment, letting yourself go into the hands of the object without any leverage. It also damages the relationship, making it irrelevant if not nonexistent: since faith is given universally, there are no expectations of the outcome, and the only possible end is disappointment the very first moment it's broken. It's binary, a gate: faith or no faith.
So, with those definitions settled, back to the case at hand: trusting an LLM as a dynamic relationship, built on judgment inferred from available data points. My current approach, though it's Saturday evening and I haven't yet had the chance to fully dig into it, would be the following. I don't have the confidence (yes) to claim this is the gold standard; it's just how I'd structure the approach.
The goal, and that's the whole point of this article, is to build trust.
1/ Data gathering
What data points can we gather that the LLM can't taint? What sources of truth do we own that don't come from the agent?
In essence: your experience and intuition, running the program, and rewriting it solo. Steps:
- Run the program in a sandbox: if it fails, that's a bad start. Are there tests? Do they pass?
- Validate the grammar independently: ask for it, check it against the actual code, fix the grammar.
- Go for a walk with the agent, line by line, and confirm your understanding of the program based on your newly acquired knowledge of the language.
- Rewrite parts of the program (critical paths, IO, gates), run them, and compare outputs. Use breakpoints if available.
- Write two or three small programs and have them run and analyzed by the agent. Does the agent flag any inconsistencies with the spec or grammar?
2/ Judgment forming
At this stage I should have a better understanding of the language and its syntax, from ingesting the grammar, writing a few simple programs, reading the tests, and mapping the program flow. From this, I can already form a few judgments: is the agent calibrated? Does it raise flags or questions when unsure about specific parts of the code? Does it stay consistent about the language spec across multiple work sessions? An agent that recognizes a lack of context or training data to properly address a question earns more trust than one that doesn't.
Judgment here is not about comparing the merits of the language with my pet language. The question, again, is whether I can trust the agent to work as a sparring partner to help me evolve and maintain not only the program but the language too. (Thinking straight away that we should just replace it with Python or X defeats the thought experiment.)
3/ Trust: balancing gains and risks
What's great about trust is that while it takes time to build, it's resilient to failure: trust can be rebuilt. Faith is given whole and lost whole.
Just like the vinaigrette, it starts with watching over every move, every measurement. A lot. Then you progressively give up some control. Giving up partial control is autonomy exercised in a delimited space. Giving up full control is taking the agent's hand and jumping into the void together. I provide the ingredients, because the task is limited to making the vinaigrette; autonomy expresses itself within the frame of that goal. But I also provide the kitchen, the utensils, a clean and safe environment, and some fresh apple juice at the end. Autonomy lives within boundaries, where the object doesn't have to worry about what happens outside them. That is taken care of without needing its intervention.
In this example, the risks feel obvious; the gains, not so much. Again, if we look at trust, as we did, through the prism of a dynamic rather than static relationship, risk isn't atomic and can be broken down into smaller areas where our trust might start to take root: areas offering a positive return relative to the risk. Here, we might realize that this language, because of its tailor-made nature, might offer ways to solve some company-specific problems in a better, faster, clearer way. A bit like all the tech insourcing programs happening inside Fortune 500 companies, but at 1e-3 of the cost.
What makes the headlines is the idea that we should have faith in the models to perform autonomously, that this is the target. Following what we wrote here, faith isn't enough to flip the switch: autonomy will arise from trust. Because trust isn't just a human trait; it's a function that shines in asymmetric, low-information configurations, something Terminator-like agents will inevitably face once given seemingly "free will" and "full autonomy."