X-Tree: Tokenizing Reusable Experience for Efficient Agent Generalization

arXiv · Sitao Cheng, Xunjian Yin, Zhiyuan Sun, Yuxuan Li, Ruiwen Zhou, Xiangru Jian, Victor Zhong · 2026-09-26

X-Tree extracts reusable action hierarchies and tests them in three agent-training settings.

What it does

X-Tree scores reusable spans in action trajectories and merges canonicalized actions into a hierarchy. Its construction recovers structure from data without an LLM-written skill stage. The tree is then used as offline-RL training instances, an online-RLVR skill bonus or privileged context for on-policy self-distillation.

Why it matters

Across WebArena, ScienceWorld and WebShop at three model scales, the authors compare these integrations with standard training recipes at matched data and budgets. This tests whether the recovered structure helps rather than merely adding more examples.

How it applies

Start with trajectories whose actions and success signals can be canonicalized. Compare a tree-based training integration with the same model, data and budget without the tree. The checked evaluation does not establish immediate transfer to robotics or every hierarchical task.

The short version

Can useful subroutines be recovered from the trajectories already collected?

My take

The first useful trial is a controlled training comparison on a domain with repeated, recognizable actions. A transferable vocabulary should be demonstrated on held-out tasks before it is called reusable expertise.

How it connects

• Action-span canonicalization
• Hierarchical reinforcement learning
• Held-out generalization at matched budgets

Bottom line

Evaluate recovered structure on a known trajectory corpus before applying it to a new domain.

Takeaways

• Data-only hierarchy recovery and model training are separate stages.
• The hierarchy is integrated into offline RL, online RLVR and self-distillation.
• The reported gains concern three named benchmark environments.
• Match data and training budget before attributing a gain to hierarchy.

Topics: agent hierarchy, skill reuse, reinforcement learning, generalization, tokenization