DIEM
About this model
Qwen 3 Next 80b is the first release in Alibaba's Qwen3-Next architecture line, an 80-billion-parameter Mixture-of-Experts model that activates roughly 3 billion parameters per token, drastically reducing FLOPs while preserving capacity. Its defining feature is a hybrid attention design combining Gated DeltaNet with Gated Attention, paired with a high-sparsity MoE and multi-token prediction for faster, cheaper inference on long inputs. Released under Apache 2.0, it ships with a native 256K context window and supports function calling and web search.
Compared with the broader Qwen3 generation, Alibaba reports meaningful efficiency gains. The team states the underlying base model reaches performance comparable to—or slightly better than—the dense Qwen3-32B while using less than 10% of its training GPU hours. On the long-context RULER benchmark, Alibaba reports that the Instruct version outperforms the earlier Qwen3 30B A3B across all tested lengths, and even surpasses the flagship Qwen 3 235B A22B Instruct 2507 within 256K context.
This positions Qwen 3 Next 80b as the efficiency-focused step in the family, trading the dense scaling of older Qwen3 models for a sparser, architecturally novel approach. Venice serves it at fp16, optimized for speed, with the weights also deployable through common engines such as vLLM and SGLang.
The model exists in two post-trained forms in Alibaba's release—an instruct variant for chat and agents and a separate thinking variant for complex reasoning—both sharing the same hybrid attention and MoE backbone. Within this catalog it is the newest entry in its architecture family, sitting alongside many other Qwen-derived text, image, and embedding siblings.
This About section is AI-generated from public sources (Claude Opus 4.8), with no human editing. It may contain inaccuracies — verify critical details against the sources listed above.
Research & Papers
4 reference papers linked from the HuggingFace model card.
YaRN: Efficient Context Window Extension of Large Language Models(2023)
Bowen Peng, Jeffrey Quesnelle, Honglu Fan et al.
Rotary Position Embeddings (RoPE) have been shown to effectively encode positional information in transformer-based language models. However, these models fail to generalize past the sequence length they were trained on. We present YaRN (Yet another RoPE extensioN method), a…
RULER: What's the Real Context Size of Your Long-Context Language Models?(2024)
Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman et al.
The needle-in-a-haystack (NIAH) test, which examines the ability to retrieve a piece of information (the "needle") from long distractor texts (the "haystack"), has been widely adopted to evaluate long-context language models (LMs). However, this simple retrieval-based test is…
Qwen3 Technical Report(2025)
An Yang, Anfeng Li, Baosong Yang et al.
In this work, we present Qwen3, the latest version of the Qwen model family. Qwen3 comprises a series of large language models (LLMs) designed to advance performance, efficiency, and multilingual capabilities. The Qwen3 series includes models of both dense and Mixture-of-Expert…
Qwen2.5-1M Technical Report(2025)
An Yang, Bowen Yu, Chengyuan Li et al.
We introduce Qwen2.5-1M, a series of models that extend the context length to 1 million tokens. Compared to the previous 128K version, the Qwen2.5-1M series have significantly enhanced long-context capabilities through long-context pre-training and post-training. Key techniques…
Data sources: Venice API · HuggingFace · Wikipedia · arXiv — enrichment updated 5d ago