Supernodes Shift AI Compute Logic from Scale to System Efficiency

Tens of cards, hundreds of cards, thousands of cards—the scale race in AI computing infrastructure is entering a new phase.

In the past, the most intuitive way for large model vendors to demonstrate their computing power was by showcasing how many GPUs they owned and the size of their compute clusters. Today, as "SuperNodes" become a hot topic in AI infrastructure, vendors have added another metric to showcase their capabilities: how many AI chips they can connect together so that they work collaboratively like a single computer.

From NVIDIA's GB200 NVL72, to Huawei's hundreds-of-cards and thousand-card-level SuperNodes, to the tightly coupled computing systems launched successively by various domestic vendors, the scale of SuperNodes continues to expand. Numbers like 72 cards, 384 cards, and 1024 cards are appearing with increasing frequency in product launches and technical discussions.

At first glance, this seems like a continuation of the previous computing power race: previously, it was about who had more GPUs; now, it is about who can connect more GPUs together.

However, what SuperNodes truly aim to solve are precisely the problems exposed by the traditional approach of expanding computing power simply by adding more machines.

As model sizes grow larger, computation is no longer the sole bottleneck. Data exchange between chips, synchronization across different computing tasks, and the efficient utilization of memory resources all begin to impact final runtime efficiency. While adding more chips certainly increases theoretical computing power, if significant time is consumed by data movement and waiting, hardware investment may not translate into corresponding performance gains.

This is why SuperNodes have attracted attention. They aim to make more chips function as a tightly coordinated computing whole through higher-bandwidth, lower-latency interconnects, and co-design of computing, memory, and software systems.

However, a question worth probing arises alongside this trend: If SuperNodes emerged to improve computing efficiency, why has card count once again become the most watched competitive metric? Does connecting more chips necessarily mean models run faster?

From "Stacking Machines" to "Reconstructing Connectivity": Why Did SuperNodes Suddenly Become Popular?

SuperNodes are not simply larger versions of past multi-GPU servers; they attempt to change how computing resources are organized.

Traditional AI clusters typically use multi-GPU servers as the basic unit. Taking common eight-GPU servers as an example, high-speed interconnect technologies such as NVLink enable data exchange between GPUs within the server. When models or training tasks require more resources, multiple servers are connected via InfiniBand or Ethernet.

This architecture offers strong scalability and supported large model training over the past few years. However, the issue lies in the fact that communication within a server and between servers does not operate at the same performance tier.

When a model can complete its primary computation within a single server, data can be exchanged using relatively high-speed internal interconnects. Once the model scale exceeds the capacity of a single server, computation and data must be distributed across more servers, frequently crossing node boundaries.

For tasks requiring extensive synchronization and data exchange, these boundaries directly impact efficiency. GPUs are responsible for executing computations on one hand, but on the other, they must wait for other chips to finish their calculations and return data before proceeding to the next step.

This means that the theoretical computing power possessed by a compute cluster is not the same thing as the effective computing power actually available to the model.

In March 2024, NVIDIA unveiled the Blackwell platform and the GB200 NVL72 at GTC, providing a striking new solution to this challenge.

The GB200 NVL72 connects 72 Blackwell GPUs into a single NVLink domain, extending the scope of tightly coupled computation from traditional multi-GPU servers to the entire rack.

Its emergence changed the organization method for scaling resources. Previously, when a model exceeded the capacity of a single server, it relied more heavily on inter-server networks. Now, more GPUs can enter the same high-speed interconnect domain, collaborating within a tighter connectivity range. What SuperNodes aim to reduce is precisely the communication and data movement overhead incurred when models cross these boundaries.

For large models, this shift holds direct appeal: more chips participating in computation also provide more VRAM resources; faster interconnects create better conditions for operations such as model parallelism and expert-to-expert data exchange. Whether these conditions can be converted into performance still depends on model partitioning, operator optimization, and scheduling, but system design now has new room for improvement.

Subsequently published model development practices further explain this demand.

In May 2025, the DeepSeek team published an architecture paper disclosing that in their H800 system, according to the paper's effective bandwidth metrics, intra-node NVLink communication was approximately 160GB/s, while inter-node InfiniBand communication was approximately 40GB/s—a difference of about four times. To control the communication costs of Mixture-of-Experts (MoE) models, the team limited the number of nodes accessed by experts per token and utilized faster intra-node connections for data forwarding.

This practice demonstrates that how chips are connected has begun to inversely influence how models are designed.

SuperNodes have attracted attention precisely because the industry needs to utilize more resources effectively while reducing the coordination burden between them.

Vendors quickly launched products along this direction.

Huawei, Inspur Information, Alibaba Cloud... almost every major vendor is positioning itself in the SuperNode space. In April 2025, Huawei Cloud released CloudMatrix 384, announcing its large-scale deployment at the Wuhu data center, pushing SuperNodes to the hundreds-of-cards scale. In July 2026, Huawei publicly demonstrated the Atlas 950 SuperPoD configured with 1024 cards, providing a 256TB globally unified memory addressing space. Recently, Inspur Information also released the YuanBrain SD200 Ultra, building a tightly coupled system with 128 domestic AI chips, offering 8TB of uniformly addressed VRAM, and optimizing around communication and model operators. It supports running the 2.8 trillion parameter Kimi K3 large model on a single machine, with token generation latency breaking below 5.85 milliseconds for the first time, equivalent to a single-user speed of 170 Tokens/s.

While different products vary in chip capabilities, interconnect mechanisms, and deployment forms, they collectively drive a shift: competition in AI infrastructure is moving from merely purchasing chips to organizing them.

Yet, after SuperNodes became popular, card count once again became the most prominent number. Tens of cards, hundreds of cards, thousands of cards—the scale of high-speed interconnect domains continues to expand, and the easiest message to propagate at launch events remains "how many cards we can connect."

What Is the True Contest Behind SuperNodes?

SuperNodes gained popularity because more chips need to collaborate more efficiently. But if "efficient collaboration" is simplified to "connecting more chips," it easily reverts to the old scale competition. After all, the ability to determine just how fast a large-parameter model can "run" is the capability that SuperNodes should truly compete on.

Taking Kimi K3 as an example, as the world's largest open-source model, K3 features 2.8 trillion total parameters and 104 billion active parameters. Official recommendations suggest that AI servers with 64 GPUs or even larger scale are required to accommodate it. However, "accommodating" does not equate to "running fast." Industry insiders note that an unoptimized supernode running Kimi K3 may initially achieve performance of only around 10 tokens/s.

This exposes another issue behind the expansion of supernode scale: Do more chips truly lead to synchronized performance growth?

72 cards, 384 cards, 1024 cards—the larger the number, the more likely it is to become the headline at launch events. For users, a larger compute domain indeed means more resources, capable of hosting larger models, storing more intermediate data, and providing space for parallel computing. The question is: What is the cost to utilize these resources? Can doubling the number of chips yield a corresponding improvement in performance?

Therefore, what supernodes really need to solve is not "how many more chips can be connected," but how to enable efficient collaboration among more chips, transforming stacked compute power into actual inference speed for globally cutting-edge models.

When asked whether a larger Scale-up domain is always better, Liu Jun, Chief AI Strategy Officer at Inspur Information, gave a direct answer: "Scale doesn't matter. The core issue is your metric: What kind of model do you want to run? Does the compute architecture matched to this model allow it to perform effectively?"

This statement must be understood within the context of his technical roadmap and product positioning, but it raises a key judgment: The value of a supernode ultimately returns to the actual performance of specific models.

Determining how many chips a model needs cannot rely solely on parameter count. Weight precision, context length, and concurrent request volume all impact memory requirements; parallel strategies and the amount of data exchanged during computation affect interconnect demands. For Mixture-of-Experts (MoE) models, one must also consider expert distribution and which chips each token needs to traverse.

Fitting a model into a single compute domain is merely the starting point for selection. Whether it achieves target latency and supports required concurrency after being fitted determines if the configuration is suitable. To make individual requests faster, more chips may be needed to process them jointly; to handle more requests, it may be more appropriate to split resources into multiple instances. These two needs do not necessarily point to the same size of compute domain.

If a model already runs efficiently within a smaller domain, continuing to expand the single domain may not be more cost-effective than adding service instances. Conversely, if a model's capacity and communication requirements exceed the limits of a smaller system, a larger tightly coupled domain may be necessary. There is no universally applicable "optimal card count" independent of the model and business context.

Scaling up also turns collaboration itself into a more complex engineering challenge.

In interviews, Liu Jun pointed out that as scale increases, communication costs and the amplification effect of issues also rise. If data and computation are too far apart, the overhead of communication and scheduling may outweigh the benefits brought by new compute resources.

High-speed interconnects can reduce these overheads but will not eliminate them. Even if resources are unified under a single addressing scheme, accessing remote VRAM does not imply the same cost as accessing local VRAM. The number of hops data traverses, link congestion, and whether computation can overlap with communication still impact actual performance. A unified address space makes resources easier to access, but software still needs to determine how to access them efficiently.

The larger the supernode, the higher the requirements for topology and software. It needs to prove not just that all cards can connect, but that resources can be flexibly allocated. When large tasks require the entire domain, can the system collaborate efficiently? When small tasks only need a portion of resources, can the remaining resources serve other workloads? Will different tasks interfere with each other? If good performance is only achievable when fully utilized, but daily loads are difficult to serve economically, the advantage of scale may turn into idle costs.

Therefore, whether a supernode is useful depends not only on how large a model it can accommodate, but also on whether the target model can achieve expected latency and throughput within it.

Reliability is another account that cannot be overlooked.

Liu Jun specifically mentioned that long-running Agent tasks may last hours or even days, and interruptions bring greater losses. However, this does not mean large supernodes are inherently unreliable, nor that a single underlying failure necessarily forces tasks to restart from scratch. The final impact depends on whether failures can be isolated, whether task states are saved, and whether services can retry, migrate, and recover.

For users, more worthwhile questions are: How much resource is affected if one link or one chip fails? Can other workloads continue running while part of the equipment is maintained? How long does recovery take, and how much progress is lost? These questions determine whether the risks of scaling can be controlled and cannot be replaced by short-term performance tests.

Finally, scale must translate into data center infrastructure and budgets. More chips require corresponding power supply, cooling, and interconnect facilities, along with software adaptation and operational maintenance costs. Procurement price is only part of the investment. If significant data center renovations or increased long-term maintenance burdens are required for limited performance gains, larger configurations may not offer better returns.

"Supernodes: The Challenge Lies in Unified Addressing"

The true appeal of supernodes lies in organizing originally dispersed compute and storage resources into a tighter collaborative whole. The problem is that "working like a single computer" is a goal description, behind which lie numerous architectural differences.

In interviews, Liu Jun particularly emphasized unified addressing. He believes that if a model is simply split into parts and run on different servers, it remains a node cluster. The supernode as understood by Inspur Information requires solving unified VRAM addressing first, enabling more direct data access between multiple accelerators.

This is an important entry point for understanding their technical roadmap, but it should not be taken directly as a definition followed by all vendors. Judging a system still requires breaking down several often-confused concepts: whether resources have a unified address representation, whether remote access can be initiated directly by accelerators, how many switching layers the access path traverses, and how software manages data placement and synchronization.

Unified addressing primarily solves the problem of "can we find it, how do we access it," but it does not automatically imply that remote VRAM has the same access cost as local VRAM, nor does it guarantee efficient sharing of all resources. Even if they appear to belong to the same address space from a software perspective, physical distance, topology, and link congestion still exist. Where data is placed, when it is moved, and whether it can overlap with computation still determine final performance.

Liu Jun used the analogy of "visiting neighbors" to explain Inspur Information's symmetric memory solution: Previously, accessing data on another GPU was like going to the property management office to get a pass; now, the goal is to go directly to the neighbor's house. He emphasized that after unified addressing, further reduction of scheduling and relay overheads in communication is still required.

The value of this metaphor lies in pulling the real problem supernodes need to solve back from "connecting more cards" to "making these cards collaborate more effectively." If the resource pool expands without reducing access and synchronization costs, the space seen by software may be larger, but the time compute units spend waiting for data may also be longer.

Therefore, "fitting the model" is only the first hurdle; "running efficiently after fitting" is the harder one.

How to Select Supernodes: Look at Business Needs First, Then Determine Scale

Another issue worth noting is that different models' system requirements do not grow proportionally with parameter count.

Weight storage requirements depend on parameter count and data precision; inference also incurs additional overheads from context length, concurrent requests, and KV Cache. For models using a Mixture-of-Experts (MoE) architecture, total parameter count, the number of parameters actually activated per token, and inter-expert communication constrain capacity, compute, and interconnect, respectively. Relying solely on claims like "supports trillions of parameters" makes it difficult to derive actual service capability.

Scale selection for training and inference should not be conflated. Training typically focuses on scaling efficiency within a single job, while inference must simultaneously balance time-to-first-token (TTFT), subsequent token generation speed, concurrency, and total throughput. Optimizing for faster processing of individual requests within a large domain may involve different strategies than partitioning resources into multiple service instances to handle more requests.

In this interview, Liu Jun categorized demand into "capability-driven" and "capacity-driven." The former supports frontier models, aiming for superior handling of complex problems; the latter serves high-volume routine tasks, seeking abundant and cost-effective intelligence delivery.

He observed that customers often require both: stronger models for exploration, planning, and critical judgments, while validated repetitive workflows benefit from faster, cheaper execution.

Enterprises need an upper bound on capability but do not necessarily require equally expensive resources for every task. Configurations capable of running frontier models should be validated for quality and completion time on complex tasks; configurations serving daily workloads should be validated for sustained request throughput under defined quality and latency constraints.

Therefore, supernode selection should start from business requirements downward: first define target model, precision, context length, concurrency, and latency requirements, then determine the necessary tightly coupled domain size based on parallelism and deployment strategies. Testing should use request lengths and arrival patterns close to real-world business scenarios, observing tail latency during peak periods rather than just the fastest generation speed under low-load conditions.

Subsequently, incrementally add resources to assess what improvements new chips actually deliver. If the model already meets requirements and expanding the single domain does not significantly improve effective throughput, reconsider resource partitioning or adding service instances. Only when bottlenecks stem from capacity limits and cross-chip communication does a larger domain offer clear value.

The optimal scale of a supernode should be derived from business testing. Vendor-provided maximum configurations indicate how far the system can scale; users must verify at which scale their specific workload achieves stable, qualified output at reasonable cost.

Supernodes emerged as the industry recognized that simply buying more chips requires solving how they collaborate. Now, as supernodes themselves enter a scale race, the industry must remain vigilant: will efficiency issues previously addressed through architectural innovation once again be obscured by larger numbers?

A sign of supernode maturity may not be "how big it can get," but "how fast it can run."

When the industry can clearly explain why adding another card is necessary—and confidently state why it isn’t—compute infrastructure development truly shifts from a scale race to a judgment of system efficiency.