DataSpace Shows Data Agents Peak at 66% — the Integration Matters More Than the Model
The DataSpace benchmark runs six frontier models and five agent implementations through 410 real-world analytical tasks across CSV, JSON, SQLite, Markdown, PDF, and video, and the best system scores 66.34% — 76 tasks stump every combination.
Why it earned a slot
The DataSpace benchmark publishes a number that should make anyone building data agents wince: 66.34% task accuracy. That's the ceiling, and it comes from six frontier multimodal models and five agent implementations on 410 tasks spanning 15.01 GB of artifacts across six formats. Seventy-six of those tasks are missed by every tested configuration. I think the 15.36-point gap between the best and worst implementation – with the same model – is the more important number. Swap the implementation, keep the model identical, and you lose fifteen points. That's not a model flaw; it's an integration issue people keep pretending doesn't exist. DataSpace was the official benchmark for the KDD Cup 2026, drawing 703 teams and 1,307 participants. So it's not a toy. The tasks require agents to discover evidence across databases, structured files, documents, and video, then produce verifiable tabular results. Execution-grounded construction, reviewed by eleven domain experts, makes the evaluation deterministic and model-free. The hardest parts? Multimodal evidence integration and cross-source joins. A model that can parse a CSV and a PDF separately doesn't automatically know how to join them by a shared timestamp column. The benchmark exposes that gap cleanly. What I don't know is how these numbers degrade with longer contexts or more heterogeneous file types. DataSpace includes video, which raises the dimension count fast. The paper does not break the universal failures down by file type, so which combinations dominate is not established here tables, but I'd need the per-task breakdown to be sure. I'm tired of watching teams celebrate a two-point bump on a new model release while their integration leaves fifteen points on the floor. DataSpace gives us a way to measure that. Use it. The paper is on arXiv, the dataset and evaluator are released. Stop chasing model upgrades and start fixing the integration. The takeaway isn’t that data agents are bad. They're promising. But 66 % on a curated benchmark means production will be lower, and an implementation swap can lose you a fifth of that. Design your evaluation around the integration, not the model.