DIEM
About this model
Hy4 Preview is the Tencent Hy Team's Mixture-of-Experts flagship, published on 28 August 2026 under the Apache 2.0 license with open weights on Hugging Face. The catalog listing describes 770 billion total parameters with 49 billion activated per token and a context window of roughly 1.05 million tokens, served in FP4 quantization.
Against its same-family predecessor, the jump is primarily one of scale: Tencent states on the model card that it scaled the model on three fronts simultaneously — model size, context length, and training data. Architecturally, the attention module uses Gated DeepSeek Sparse Attention with IndexCache for cross-layer sparse index reuse, while the residual pathway adopts identity Hyper-Connections to widen inter-layer information flow.
Positioning is agentic and work-oriented rather than general chat. The model is post-trained for software engineering, office and data analysis, game development, and scientific research, with emphasis on long-horizon reasoning and tool use; listed capabilities include reasoning, code optimization, function calling and web search. The very large context window is the natural complement to that focus, supporting whole-repository or multi-document sessions.
Tencent frames this as a deliberately early release, writing on the model card that it would "rather ship early and hear what breaks," the same iterative approach it credits for improving the prior Hy generation.
Sources: [1] tencent/Hy4-preview model card, Hugging Face.
This About section is AI-generated from public sources (Claude Opus 5), with no human editing. It may contain inaccuracies — verify critical details against the sources listed above.
Research & Papers
2 reference papers linked from the HuggingFace model card.
DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models(2025)
DeepSeek-AI, Aixin Liu, Aoxue Mei et al.
We introduce DeepSeek-V3.2, a model that harmonizes high computational efficiency with superior reasoning and agent performance. The key technical breakthroughs of DeepSeek-V3.2 are as follows: (1) DeepSeek Sparse Attention (DSA): We introduce DSA, an efficient attention…
IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse(2026)
Yushi Bai, Qian Dong, Ting Jiang et al.
Long-context agentic workflows have emerged as a defining use case for large language models, making attention efficiency critical for both inference speed and serving cost. Sparse attention addresses this challenge effectively, and DeepSeek Sparse Attention (DSA) is a…
Data sources: Venice API · HuggingFace · Wikipedia · arXiv — enrichment updated 1h ago