RAG's Trust Problem: 91% Accuracy Hides the Poisoning You'll Miss
Trustworthy RAG reaches 91% accuracy on TruthfulQA by detecting knowledge poisoning before generation, but in-place edits and semantic weakening remain undetectable—exposing why retrieval-grounded systems still trust the wrong sources.
Why it earned a slot
The Trustworthy RAG paper (arxiv 2608.21095) landed this week with a number that sounds like a solution: 91% accuracy on TruthfulQA with Llama 3.3 70B. Read the fine print and you find the actual problem. The system catches instruction injection with 100% recall and 100% precision. It fails silently on entity swaps and subtle semantic shifts. This is not a bug report—it's a confession about what detection can and cannot do. I think the paper is honest in a way that matters. Most RAG work pretends the retrieval layer is clean. Trustworthy RAG admits it isn't, then builds a five-signal poison detector that combines Natural Language Inference factual verification with a Trust Index formula: 0.4 times factuality, plus 0.35 times consistency, minus 0.25 times the poisoning probability. It weights these signals by relevance. The result is a middleware layer that sits between retrieval and generation and refuses to pass contaminated context forward. But here's what I don't know: how many real-world poisoning attacks look like the instruction injections they test for. The paper shows that across three LLMs, the Trust Index stays discriminative with ROC-AUC between 0.73 and 0.81. That's not baseline—it's real separation. Except when it isn't. Entity swaps—where an attacker replaces a person or place in a retrieved document—remain hard to detect. So do contradictions and what the paper calls "subtle semantic weakening," which is the kind of drift that doesn't trigger NLI but shifts the answer anyway. The software-engineering use case is where the system earns its skepticism. A secure-coding assistant that retrieves guidance from OWASP Top 10 and CWE reaches F1 92% on blocking instruction injection. That's the hard case—explicit malicious advice. Contradiction and semantic weakening still get through. If you're using this in production, you're catching the attacks that announce themselves and missing the ones that don't. What matters is that the authors didn't hide this. They ran cross-dataset generalization and found it requires domain-specific calibration—meaning a Trust Index trained on one domain's poisoning patterns doesn't transfer. Per-LLM threshold calibration restores baseline competitive accuracy, but that's overhead most teams won't absorb. The FEVER result is weaker than TruthfulQA, suggesting the detector's confidence is dataset-specific. The real insight is that detection-before-generation is not the same as prevention. You can measure the poisoned context before the model sees it, which is useful. But you cannot measure what the model will do with context that passes your filter. If your detector misses a semantic shift, the LLM will still integrate it. If the shift is subtle enough, you may never know. I'd run this in front of any RAG system that serves high-stakes queries. It's a real constraint on what retrieval can poison you with. But read it as a lower bound on what gets through, not an upper bound on what you've caught.