Today for AI
HOT RADAR
llmHEAT 14.3°

AI CURATED HOT STORY · 11/3/2026

Zyphra Introduces ZHA Hierarchical Attention for Efficient Long-Context Inference

1 reports archived1 independent sourcesupdated 11/3/2026, 09:13:39
Synthesis & Latest Updates
1 Sources Cross-Validated

Zyphra introduces ZHA (Hierarchical Attention), a sparse attention method utilizing a tree structure over the KV cache for efficient retrieval. On Qwen3.6-35B, it outperforms dense attention on LongBench v2 while reading only ~1.1% of keys, with minimal training overhead (optimizing just 80k parameters). Although trade-offs exist in specific benchmarks like Terminal-Bench, it offers a novel architectural approach to long-context processing.

LATEST/Zyphra introduces ZHA (Hierarchical Attention), a sparse attention method utilizing a tree structure over the KV cache for efficient retrieval. On Qwen3.6-35B, it outperforms dense attention on LongBench v2 while reading only ~1.1% of keys, with minimal training overhead (optimizing just 80k parameters). Although trade-offs exist in specific benchmarks like Terminal-Bench, it offers a novel architectural approach to long-context processing.

Hourly heat curve

4 fully observed hours
Now
14.3
Peak
15.210:00
24h change
–

The line compares only sources observed throughout; gap hours are interpolated to keep the trend continuous.

Coverage timeline

Total 1 reports · Latest first
  1. Marktechpost (@Marktechpost)T2·78 pts

    Zyphra Introduces ZHA Hierarchical Attention for Efficient Long-Context Inference

    • ZHA uses a tree structure over the KV cache, employing hierarchical compression and top-p routing for sparse retrieval.
    • Outperforms dense attention on LongBench v2 while reading only ~1.1% of keys; RULER-13 scores are comparable to dense mode.
    • Conversion requires training only 81,920 parameters with ~4B tokens of KL distillation, indicating low deployment cost.