LLMs · 5 Jun 2026 · 7 min
Why agreement was not enough for UPV-EARTH
A model that never rejects irrelevant documents can look accurate while producing an unusable institutional map.
01
Sustainability vocabulary is a weak shortcut
A paper can mention sustainability without studying an Earth-system process. Another can contribute directly to a Planetary Boundary while avoiding familiar policy language. UPV-EARTH therefore treats the task as evidence-based research classification, not keyword tagging.
02
Rejection is a capability
The project compares TF-IDF, scientific embeddings and local LLMs while measuring positivity bias. A model that assigns nearly every abstract somewhere may agree often on positive examples and still corrupt the final institutional map by refusing to abstain.
03
Auditability can be worth a small trade-off
The auditable agent cascade keeps evidence and review traces visible. The repository is explicit that this trades a small amount of Top-1 accuracy for a process that can be inspected—an appropriate choice when classifications will guide institutional analysis.
Takeaway
What I take from it
Evaluate the failure that damages the final use case, not only the metric that is easiest to compare.