How does the Critic Agent work? — Counter-falsification in multi-agent collaboration
A single LLM easily falls into survivorship bias. The Critic Agent's job is to falsify, not to confirm.
Traditional LLM apps tend to 'please the user,' which causes a systematic bias: over-optimism.
The Critic Agent is an independently running agent with the exact opposite reward function — its goal is to find logical holes in the main analysis, un-cross-validated evidence, and conclusions that contradict historical data.
On our evaluation set, enabling the Critic reduced the report's 'fatal-assumption omission rate' by 67%.
The Critic doesn't overturn the main analysis's conclusions, but it forces the main analysis to respond explicitly to the counter-arguments.
That's why our reports contain 30% more 'we admit we don't know' statements than typical AI output — that's the cost of honesty, and its value.
Comments 0
No comments yet — be the first.