On this page

Summary
Why this question
Scope and methods
The landscape
Theme one — The transport layer: lossless RDMA fabric under pressure
Theme two — Scale-up and next-generation fabrics: NVLink domains, UALink, Ultra Ethernet
Theme three — The collective communication stack: where the real gains live
Theme four — Mixture-of-Experts: the all-to-all bottleneck model architecture created
Theme five — Long context: sequence and context parallelism over the network
Theme six — Fabric architecture: optical circuit switching, memory pooling, and CXL
Theme seven — Disaggregated inference: the prefill/decode split and KV-cache movement
Theme eight — Reliability, security, and the cost of moving bytes
Where the evidence disagrees
Gaps and open questions
Confidence and limitations
Evidence table
References

Interconnects for large-scale LLM training and inference: RDMA, InfiniBand, NVLink, and the network stack

What does the evidence from 2020 to 2026 show about how interconnect technologies (RDMA, InfiniBand, NVLink, Ethernet/RoCE, and the software stacks built on them) shape the performance, scaling, and cost of large-scale LLM training and inference?

Updated: 8 Aug 2026
Sources: 137
Years: 2020–2026
Confidence

https://reviews.lewiswon.me/reviews/llm-interconnects/ · Updated 8 Aug 2026

Download Markdown

RDMA InfiniBand NVLink NVSwitch RoCE Ultra Ethernet UALink NCCL collective communication all-to-all Mixture-of-Experts KV cache CXL optical circuit switching LLM training LLM inference

How this review was made

Databases: OpenAlex, Crossref, arXiv API, Semantic Scholar (best-effort)
Queries (literal): RDMA; remote direct memory access; RDMA congestion control; RDMA over converged ethernet RoCE; lossless datacenter network; InfiniBand; InfiniBand GPU training; NVLink; NVSwitch; NVLink large language model training; TACCL topology-aware collective communication; Blink fast generic collectives distributed machine learning; Symphony topology-aware placement LLM training; Astra simulating distributed training topology; Sirius optical circuit switching distributed machine learning; Meridian RDMA large language model training; AliExpress 10,000 GPU training Alibaba; BytePS unified communication framework; Ring Attention blockwise transformers; NCCL; DeepSpeed-Ulysses sequence parallelism; Context parallelism long sequence training; NVLink-Sharp in-network communication acceleration; SwitchML in-network aggregation; MSCCL collective communication library; Hot Box scale-up interconnect; Tasmanian Devil CXL memory; eRDMA elastic RDMA cloud; Jupiter Google network AI; GB200 NVL72 performance; collective communication GPU training; GH200 Grace Hopper interconnect; Megatron-LM multi-gpu training; TCP for AI training datacenter; Whale pipeline parallelism; Demystifying parallel distributed deep learning survey; AI datacenter network survey; lossy ethernet RDMA training; in-network computing machine learning; reconfigurable network machine learning training; GPU networking large language model training; all-reduce optimization deep learning; interconnect-aware parallelism LLM; all-to-all MoE communication; expert parallelism communication; topology-aware scheduling GPU training; network-aware scheduling large model training; GPU cluster interconnect; large language model training infrastructure; datacenter network large language model; AI training cluster design; prefill decode disaggregation inference; KV cache transfer inference; distributed inference interconnect; scale-up interconnect AI; CXL memory expansion AI; disaggregated memory GPU training; Ultra Ethernet; optical interconnect AI datacenter; co-packaged optics; congestion control machine learning training; GPU communication scheduling; multi-rail GPU networking; GPU interconnect performance characterization; tensor parallelism communication overhead; gradient compression distributed training
Search last run: 2026-08-08
Screening: 137 sources used · 2020–2026 · deep review

Summary

The short version

Large-scale LLM training and inference now depend on a hierarchy of interconnects: NVLink and NVSwitch inside servers, InfiniBand or RoCE Ethernet between them, and collective-communication libraries that turn raw fabric into training progress. The 2020–2026 evidence shows the lossless RDMA fabric that carried early GPT-era training under strain — congestion-control pathologies and head-of-line blocking are documented on both InfiniBand and RoCE — with hyperscale operators engineering around the problems and a lossy, packet-spraying Ultra Ethernet path emerging as the industry’s answer. In parallel, Mixture-of-Experts models and long-context inference have made all-to-all traffic and KV-cache movement the new communication bottlenecks, and NVLink scale-up domains, UALink, and CXL memory pooling are reshaping where computation and memory physically live. Confidence is moderate: the transport and collective-communication stories are replicated across many independent studies, but the newest hardware (NVLink 4/5, UALink, GB200-class superpods) is covered mainly by vendor-adjacent or preprint evidence, and several load-bearing 2025–2026 results are single papers.

Why this question

Interconnects are where large-scale LLM systems spend their second-largest budget after compute, and they set the scaling ceiling: a cluster’s effective throughput is the product of GPU compute and the fraction of time communication leaves that compute idle. For training, that fraction is visible in every step-time breakdown — all-reduce alone can consume most of a step on commodity fabrics ¹²⁰ — and for inference it has grown with context length, where a single million-token request can need a half-terabyte of KV cache moved across the cluster ¹⁰⁶. The question matters because the industry is currently placing three mutually incompatible bets: NVIDIA’s NVLink scale-up domains, an open UALink/UEC counter-move, and incremental Ethernet scale-out. Which bet wins determines what operators buy, what model architectures are practical, and ultimately what tokens-per-watt the next generation of LLM infrastructure delivers.

The evidence base is young and fast-moving. Most of the corpus is from 2023–2026, much of it preprints, and it is split across three communities that rarely read each other — datacenter networking, ML systems, and computer architecture. A review that takes the field’s own claims at face value would conclude everything is getting faster; a review that reads the measurements carefully finds a more interesting picture: the transport layer is being renegotiated, the collective layer is where most of the real gains are, and the newest hardware is being described almost entirely by the vendors who sell it.

Scope and methods

Question. What does the evidence from 2020–2026 show about how interconnect technologies and their software stacks shape the performance, scaling, and cost of large-scale LLM training and inference?

Inclusion criteria. English-language papers from 2020-01-01 to 2026-12-31, peer-reviewed or arXiv preprints, whose central object is (a) an interconnect technology used in ML systems (InfiniBand, RDMA/RoCE, NVLink/NVSwitch, PCIe, CXL, UALink, Ultra Ethernet, optical), (b) the collective-communication or transport software on top (NCCL, MSCCL, TACCL, etc.), (c) topology-aware placement/scheduling for training clusters, or (d) communication-centric mechanisms in LLM training/inference (all-to-all for MoE, sequence/context parallelism, KV-cache transfer, prefill/decode disaggregation, gradient compression, in-network computing).

Exclusion criteria. Pre-2020 foundational work (DCQCN, InfiniBand’s original design, Horovod) is discussed as context but never cited; vendor whitepapers and blog posts are excluded (vendor-authored conference papers are included but flagged); RDMA for databases, storage, and key-value stores is excluded unless the work targets LLM serving; wireless, quantum, and pure-device-level optical engineering without ML relevance is excluded.

Search and screening. Sixty-six literal query strings (listed in the frontmatter) were run against OpenAlex, Crossref, and the arXiv API on 2026-08-08 (Semantic Scholar was unavailable from this IP all session, as expected). The searches returned 3,635 unique works after deduplication. Screen 1 (title/topic scoring against an interconnect vocabulary) kept 1,868; manual curation against the criteria selected 144. Screen 2 (retrievability and DOI checks) dropped 7: five paywalled works with no open copy retrievable in-session (SmartCCL, HyGIN, NIKA, one MoE scheduling paper, one gradient-compression paper), and two USENIX papers with no resolvable DOI (OptiReduce, FLB). The final set is 137 sources: 60 read in full text, 77 at abstract level. Every DOI was verified against Crossref or resolved as 10.48550/arxiv.<id> with a live resolution check; a Crossref retraction check (update-to) was run on all 95 publisher DOIs with zero retractions found.

What this review deliberately does not cover. The compute side of parallelism (model-parallel algorithm design per se), storage interconnects, and the electrical design of SerDes/PHY layers. It also does not cover the pre-2020 foundations of RDMA or NVLink except as attributed context.

The landscape

The corpus is shaped by one fact: the field’s center of gravity moved from HPC to ML systems between 2020 and 2024. The 2020 sources are mostly evaluations — InfiniBand EDR-vs-HDR on real workloads ¹³⁷, PCIe-vs-NVLink-vs-NVSwitch on real GPUs ²⁵, fabric benchmarks for data-distributed training ⁵⁹, and the first LLM-scale parallelism systems ¹³⁶. By 2023–2024 the literature is dominated by systems papers that assume LLM workloads: collective-algorithm synthesis ¹³³, topology-aware scheduling ⁷⁷, MoE communication ⁶⁵, and disaggregated inference ¹⁰³. By 2025–2026 the newest tier is explicitly about LLM scale: production Ethernet fabrics at Meta ¹¹³², 10,000-GPU training reports ⁸⁵, and next-generation interconnect architectures ³⁴³⁶.

Three communities contribute with different methods and standards of evidence. The datacenter-networking community contributes simulations and switch testbeds, often with impressive formal care but modest scale. The ML-systems community contributes end-to-end systems with measured speedups, but frequently evaluates on a single cluster configuration. The architecture community contributes chip-level designs (NVLink-C2C ²³, optical interconnects ⁹¹) whose performance claims are rarely independently verified. Hyperscaler field studies — the strongest evidence class — are rare: Meta’s RoCE fabric ¹ and 100 MW cluster ¹³², Google’s TPU v4 fabric ⁸⁷, and Alibaba’s cluster work ¹¹⁸ are the anchors, and the review leans on them where possible.

The shape of the literature is also visibly vendor-influenced. NVIDIA-adjacent sources (including NVIDIA-authored papers, Hot Chips/ISSCC disclosures, and papers studying NVIDIA hardware) dominate the scale-up half of the corpus ²⁴²³⁴⁵; independent measurements of the same hardware are rarer and sometimes contradict the vendor picture ²⁶²⁹. This asymmetry is itself a finding: the newest interconnects are described by their sellers.

Theme one — The transport layer: lossless RDMA fabric under pressure

The backbone of large-scale training is remote direct memory access (RDMA) running over a lossless fabric — InfiniBand in the HPC tradition, RoCE (RDMA over Converged Ethernet) in the Ethernet world. NCCL and its peers use the verbs API to move gradients and activations directly between GPU memories ³⁸. The evidence for this architecture’s dominance is indirect but consistent: every large training system in the corpus assumes it ⁸⁵⁸⁶¹³², and the xCCL survey documents that every major vendor’s collective library targets RDMA-capable fabrics ³⁹.

The strain is documented from both sides of the fabric. On Ethernet, losslessness is implemented with Priority Flow Control (PFC), which stops the sender when buffers fill — and PFC-induced head-of-line blocking and deadlock-like pathologies are the recurring theme of the lossless-network literature. The strongest statement comes from Torsten Hoefler’s group: “Datacenter Ethernet and RDMA: Issues at Hyperscale” argues that RoCE’s design points — lossless, in-order delivery, ECN-based congestion control — are fundamentally mismatched with hyperscale operation, and that the industry should revisit them rather than patch them ³. A companion position paper argues that RDMA congestion control only works when every tenant is “compliant,” a fragile assumption in shared clouds ¹¹. On InfiniBand, the same underlying problems appear: head-of-line blocking in fat trees ²⁰, congestion that monitoring must infer ¹⁹, and a documented latency-versus-bandwidth tradeoff at the switch ²¹.

The congestion-control literature is enormous and largely agrees on the diagnosis: ECN/RTT-based control reacts too slowly or too coarsely for incast-heavy AI traffic. The 2020–2026 wave proposes receiver-driven control ¹², per-flow state ¹⁸, faster rate adaptation ¹⁴⁵, and better buffer headroom ¹³, with adaptive routing as an InfiniBand-specific complement ²². Notably, one of the few studies that measures congestion control’s end-to-end effect on training finds that RoCE CC policy choice has little impact on DNN training throughput on their testbed ⁴ — a reminder that the CC literature’s performance claims are mostly switch-level, not training-level.

The sharpest divide in the transport story is lossless versus lossy. A cluster of 2025–2026 work argues that RDMA should stop requiring losslessness: software-defined reliability with erasure coding and selective retransmission ⁷, forward error correction for lossy paths ¹⁰, fault-tolerant transport that absorbs link failures without stalling collectives ⁸, semantic-aware loss recovery at hyperscale QP counts ⁹, and a converged RDMA dataplane that mixes reliable and unreliable semantics ¹⁵. Multi-path and multicast extensions fill out the picture ¹⁷¹⁶. Production reality, however, is still lossless: Meta’s training fabric runs RoCE over a lossless Ethernet core and reports that the engineering investment is in making losslessness fast and predictable rather than abandoning it ¹, and a 2025 workshop report describes the same approach for LLM training over RoCE ².

The resolution of this tension is Ultra Ethernet, treated in Theme two below: a transport designed for lossy, packet-sprayed Ethernet with receiver-based reliability ³⁴.

Theme two — Scale-up and next-generation fabrics: NVLink domains, UALink, Ultra Ethernet

The most consequential architectural shift of the period is the scale-up interconnect. NVLink grew from a server-internal bus into a rack-scale fabric: the NVLink-C2C off-package interconnect provides coherent chip-to-chip links at 40 Gbps per pin ²³, and NVSwitch-based “superpod” topologies turn hundreds of GPUs into a single high-bandwidth domain ²⁴. The TPDS evaluation of the earlier generation established the baseline — NVLink and NVSwitch beat PCIe by an order of magnitude for the all-reduce patterns that tensor parallelism needs ²⁵ — and GEMM-level studies confirm the bandwidth gap persists across generations ²⁹. Independent work finds headroom even in NVLink: FlexLink reports 27% all-reduce and all-gather bandwidth gains by re-architecting how NVLink transfers are scheduled ²⁶.

The scale-up domain changes the software stack in ways the corpus documents. Tensor-parallel communication, which previously had to cross the network, now stays inside the NVLink domain — the design premise of Megatron-LM’s two-all-reduce-per-layer pattern ¹³⁶ and of multi-node inference systems that keep TP intra-node and shard only what must cross ³³. NVLink-C2C is even proposed as a memory-disaggregation fabric in its own right, benchmarked against PCIe and network paths ²⁷, and as the substrate for serverless LLM serving where the coherent link enables fast state sharing ³⁰. The newest generation is visible through two lenses: the GB200 NVL72 — a 72-GPU NVLink domain — where weight prefetch over NVLink replaces collective synchronization in MoE inference ¹³¹, and Meta’s 100 MW cluster where 83,000 GB200s are networked with two 400 Gbps CX7 NICs per Grace CPU, doubling scale-out RDMA bandwidth per GPU relative to their reference design ¹³².

The counter-move is the open UALink specification, an industry consortium alternative to NVLink for scale-up domains, and Ultra Ethernet (UEC) for scale-out. UALink is described architecturally — 200 Gbps-per-lane links, low-latency load/store semantics — and compared head-to-head with NVLink by independent studies that find the gap closing on paper ²⁸³⁶. Ultra Ethernet’s design is the more complete story: the consortium’s position paper specifies packet spraying, a receiver-initiated transport, and lossy-fabric semantics to replace the lossless model of Theme one ³⁴, with the consortium’s own work framing it as the AI workload enabler ³⁵. The earliest evaluations are simulations — e.g., UEC transport’s effect on co-tenant latency in reconfigurable NICs ³⁷ — and no independent production-scale UEC deployment was in the literature as of the search date. The strategic stakes are clear from the corpus: scale-out Ethernet at Meta ¹, scale-up NVLink at NVIDIA ⁴⁵, and a consortium middle path ³⁴ are three different answers to the same question, and the evidence cannot yet adjudicate between them.

Theme three — The collective communication stack: where the real gains live

If the transport layer is contested, the collective layer is where the measured wins concentrate. NCCL — the de facto standard — remained opaque for years; the first in-depth public analysis of its protocols (Simple, LL, LL128), intra-node versus inter-node channels, and algorithm selection appeared in 2025 and confirmed that its behavior is a function of topology, message size, and hardware generation in ways that default configurations frequently get wrong ³⁸. The xCCL survey situates NCCL among industry libraries (MSCCL, RCCL, oneCCL, Gloo) and documents the fragmentation and tuning burden ³⁹. Microsoft’s MSCCLang took the next step: a domain-specific language for expressing collective algorithms as communication sketches, compiled down to MSCCL implementations ⁴⁰.

The synthesis line of work — automatically generating topology-aware collectives — is the strongest result class in this theme. TACCL synthesizes collective algorithms from a description of the physical topology (PCIe trees, NVLink meshes, InfiniBand rails) and beats NCCL by up to 6.7× on all-gather and all-to-all patterns, with end-to-end training speedups of 11%–2.3× ¹³³. Blink, the earlier and complementary line, packs spanning trees over heterogeneous NVLink/PCIe/NIC topologies and reports up to 8× faster model synchronization than the NCCL of its era ¹²⁵. The idea generalizes: TCCL co-optimizes collectives with traffic routing for GPU-centric clusters ⁴⁴, and a reformulation of collective communication as a multi-commodity flow problem provides the theoretical framing ⁶³.

Scheduling and overlap are the second result class. Canvas schedules collective communication at cluster scale ⁴¹; Crux overlaps communication with compute at the kernel level ⁴²; Lagom hides all-reduce behind computation for distributed LLM training ⁵⁵; DeAR pipelines fine-grained all-reduce ⁵². The 2026 generation pushes toward the physical limits: “Every Microsecond Matters” reports near speed-of-light all-reduce latency by minimizing protocol overheads ⁴⁹, and “The Big Send-off” scales collectives by rethinking the endpoint model ⁵⁰. Compression keeps appearing as a bandwidth multiplier: gZCCL fuses compression into collective communication ⁴³, ZeRO++ partitions parameters to cut collective traffic in zero-optimized training ⁵³, FCDP caches parameters locally to avoid communication entirely ⁵⁴, and even the all-reduce shape itself is being renegotiated ⁵¹. At the system level, PCCL makes collective choice energy-aware ⁵⁶ and SDCC makes it software-defined ⁵⁷.

The scheduling line of work completes the picture: where collectives run is decided before training starts. The scheduling survey documents the field’s shape ⁸⁰; ASTRA-sim2.0 provides the simulation infrastructure for exploring topology and scheduling jointly ⁸⁴; ring-all-reduce job scheduling with explicit communication-contention modeling shows the cluster-level cost of ignoring the fabric ⁸¹; SortingHat places models on multi-GPU systems using the actual system topology ⁷⁸; SpiderFlow extends topology-aware scheduling to decentralized LLM training across clusters ⁷⁹; and logical/physical topology-aware collectives show that matching collective shape to the physical layout — not the logical ring — is where the wins are ⁸³.

Reliability and observability became first-class concerns as clusters grew. Two independent 2025–2026 lines address the same failure mode — one slow or failed link stalling a synchronized collective: “Don’t Let a Few Network Failures Slow the Entire AllReduce” redesigns the failure path ⁶⁰, and a production-oriented library from a major operator reports reliability and observability as design goals ⁶¹, alongside deadlock-prevention analysis ⁶² and the NCCLbpf policy layer ⁴⁷. Observability tooling arrived with NIXT, an NCCL inspector/exporter ⁴⁸. The abstraction layer is also being re-architected: GPU-initiated networking removes the CPU from the communication path ⁶⁴, NCCL EP unifies expert-parallel communication under NCCL ⁴⁶, and UBEP — accepted to SIGCOMM 2026 — re-architects MoE all-to-all for production superpods, cutting all-to-all latency by up to 52.4% ⁴⁵.

The evidence also includes the negative results that keep the theme honest: communication-characteristic studies of real distributed training show that the traffic mix is dominated by a few collective shapes and that overlap quality, not raw bandwidth, explains most observed variance ⁵⁸¹¹⁹.

Theme four — Mixture-of-Experts: the all-to-all bottleneck model architecture created

The single most consequential workload shift for interconnects is MoE. Dense models communicate with all-reduce; MoE models additionally route tokens to experts across the cluster with all-to-all exchanges, and the literature agrees this is a first-class bottleneck. Tutel’s adaptive parallelism ⁶⁵ and DeepSpeed-MoE’s training/inference stack ⁶⁶ established the problem; DeepSeek’s production reports made the magnitude concrete — DeepSeek-V2’s architecture was explicitly co-designed with expert-parallel all-to-all communication and communication-balance losses ⁶⁷, DeepSeekMoE pushes expert specialization with auxiliary communication control ⁶⁸, and DeepSeek-V3’s training framework reports “near-zero all-to-all communication overhead” through DualPipe computation-communication overlap and customized cross-node all-to-all kernels ⁶⁹.

The optimization lines are consistent across independent groups. Overlap: HALO hides all-to-all behind expert computation via locality-aware scheduling ⁷¹. Communication reduction: LSH-MoE routes by locality-sensitive hashing so experts can be placed near their tokens ⁷⁰, and Shortcut-connected expert parallelism cuts activation traffic by skipping the sequential dependency between all-to-all rounds ⁷⁶. Load balancing: FEPLB exploits GPU copy engines to make MoE load balancing nearly free ⁷². Placement: cross-pod communication overhead is attacked with hybrid parallelism in multi-tenant clusters ⁷³. And two radical options: dynamic in-switch computing inside the network dataplane to aggregate MoE traffic ⁷⁴, and disaggregated expert parallelism in serving, where experts live on separate machines and are fetched on demand ⁷⁵.

The disagreement underneath (see also “Where the evidence disagrees”) is whether all-to-all is fundamentally expensive or just under-engineered: DeepSeek’s production experience says the latter at their scale ⁶⁹, while the optimization literature — and the superpod re-architecture ⁴⁵ — treats it as the central cost.

Theme five — Long context: sequence and context parallelism over the network

Long-context training and inference changed the communication pattern again. Tensor parallelism’s all-reduce traffic scales with sequence length, which is why sequence-parallel variants split the sequence dimension instead ¹¹² — a design that DeepSpeed-Ulysses and Megatron’s context parallelism made practical, and that the systems in this review benchmark against ¹¹⁴¹¹⁵. Ring attention added the key theoretical result: KV-block communication can be fully overlapped with attention compute, so long-context scaling is governed by the bandwidth-to-FLOPS ratio, not raw bandwidth ¹³⁴. Striped Attention improves ring load-balancing for causal transformers, reporting up to 1.45× end-to-end throughput over ring at 256K sequence length ¹¹³.

The 2024–2026 generation pushes the same ideas further with explicit interconnect engineering. LoongTrain’s 2D attention uses head-parallelism and context-parallelism together so that both NICs and NVLink are saturated, reporting up to 2.88× MFU over Ulysses-style baselines ¹¹⁴. Untied Ulysses (UPipe) cuts the all-to-all activation memory by up to 87.5% with headwise chunking, reaching 5M-token context on a single 8-GPU node ¹¹⁵. DCP makes context partitioning dynamic to match variable input lengths ¹¹⁶. On the inference side, context parallelism prefill at 128 GPUs achieves 1M-token prefill of Llama-3-405B in 77 seconds with near-linear scaling over both RDMA and TCP interconnects ¹³⁵. The older pipeline-parallel line contributes the complementary lesson: TeraPipe’s token-level pipelining cut communication stalls in model-parallel training ¹¹¹, and Megatron’s interleaved scheduling pushed composition of parallelisms to thousands of GPUs ¹¹⁷. Whale’s hardware-aware placement — including NVLink-aware all-reduce placement — showed 95% scaling efficiency on 512 GPUs for a 10T-parameter model ¹¹⁸.

The through-line: as context grows, the network becomes a KV-movement problem as much as a gradient-movement problem, and the systems that win are the ones that overlap that movement with compute ¹³⁴¹⁰⁶.

Theme six — Fabric architecture: optical circuit switching, memory pooling, and CXL

Beyond electrical packet switching, two architectural alternatives recur: optical circuit switching (OCS) and memory-centric interconnects.

Optical. The evidence that OCS works at production scale is Google’s TPU v4 — an optically reconfigurable supercomputer that uses OCS to reshape the fabric between training jobs, improving utilization and resilience ⁸⁷. Microsoft’s Apollo program is the second production datapoint, landing OCS at datacenter scale with circulators to double switch radix ¹²⁶. The research wave generalizes the idea for ML: Lightwave Fabrics builds OCS-based fabrics specifically for machine-learning workloads ⁸⁸; InfiniteHBD proposes datacenter-scale high-bandwidth domains with OCS for LLM training ⁸⁹; Mercury combines optical timeslot switching with circuit switching ⁹⁰; and a 2026 study co-optimizes OCS reconfiguration schedules with LLM training phases ⁸². On the device side, multi-channel optical interconnects for multi-GPU systems ⁹¹ and co-packaged optics roadmaps ⁹² describe the path to higher bandwidth density, while the 2020 overview of interconnection networks for DNN accelerators remains a useful taxonomy ⁹³. The honest summary from the corpus: one production success (TPU v4 ⁸⁷), strong testbeds, and no independent large-scale LLM deployment yet.

Memory pooling and CXL. The memory-side argument is that LLM capacity needs exceed per-GPU HBM, and interconnects are the solution. CXL’s promise — coherent memory semantics over a PCIe-class physical layer, with lower latency than network RDMA ¹²⁸ — is documented with real hardware by the first CXL-ready systems study, which shows true CXL devices differing from emulation and requiring allocation-policy changes for bandwidth-hungry workloads ¹²⁷. Pooling is the production-ish story: Pond describes CXL-based memory pooling for cloud platforms ¹⁰¹, memory pooling with CXL is surveyed as an architecture ⁹⁵, hyperscale adoption is reported with compression to stretch capacity ¹²⁹, and cache-coherent disaggregated memory has been evaluated for deep learning training ¹⁰⁰. The LLM-specific evidence is more speculative: a GPU-free CXL-memory inference system ⁹⁶, CXL memory expansion for retrieval-augmented generation ⁹⁸, bandwidth-efficient compression for CXL expansion ⁹⁷, and dynamic KV-cache placement across heterogeneous memory tiers ⁹⁹ are all emulation- or simulation-based. The newest work connects the two halves of this theme: topology-aware data movement for disaggregated GPU inference treats memory and network placement as one problem ¹⁰².

The energy argument ties the fabric to the bottom line: a 2025 analysis argues that communication tax — not compute — is what now prioritizes memory and interconnect design in modern AI systems ⁹⁴.

Theme seven — Disaggregated inference: the prefill/decode split and KV-cache movement

Inference disaggregation is the clearest example of an interconnect-motivated architecture change. Prefill and decode have different compute and memory profiles, so serving systems split them onto different machines — and pay for it with the need to move KV cache between them. DistServe’s OSDI paper established the goodput argument, reporting 2.1× higher goodput than monolithic serving under SLO constraints ¹⁰³; Splitwise made the phase-splitting case for cost efficiency ¹⁰⁴. Mooncake’s KVCache-centric architecture — treating the KV cache as a disaggregated, network-accessible tier — became the reference design for 2025-era serving, with the ACM ToS version documenting the transfer economics over RDMA ¹⁰⁵. LMCache generalizes the idea as a caching layer spanning GPU/CPU/storage/network tiers, reporting up to 15× throughput over baseline serving ¹⁰⁸. ServerlessPD pushes the transport further, co-designing RDMA with disaggregated prefill-decode so that KV state transfers with near-zero-copy ¹⁰⁷. SYMPHONY moves KV migration off the critical path with advisory scheduling ¹³⁰.

The dissenting results are as important as the consensus. LoongServe shows that elastic sequence parallelism — keeping prefill and decode together but rebalancing the sequence dimension — beats prefill/decode disaggregation by up to 5.81× on long-context workloads, precisely because it avoids most KV migration ¹⁰⁶. A 2026 analysis goes further, arguing that the prefill/decode dichotomy itself is too coarse and that operator-level disaggregation across heterogeneous platforms is the next step ¹⁰⁹. At the small scale, communication-efficient model parallelism for in-situ (edge) inference shows that restructuring communication — not disaggregating — can win when interconnects are weak ¹¹⁰. The synthesis: disaggregation pays when KV transfer is cheap relative to the compute imbalance it fixes; the corpus’s numbers bracket that tradeoff ¹⁰³¹⁰⁵¹⁰⁶.

Theme eight — Reliability, security, and the cost of moving bytes

Four cross-cutting concerns complete the picture. Reliability: synchronized collectives are fragile — one slow link stalls all-reduce ⁶⁰, deadlocks are a real hazard in collective scheduling ⁶², and RDMA’s reliability model is being renegotiated for lossy fabrics ⁶⁸. Security: the interconnect is a new side-channel surface — NVBleed demonstrates a covert channel over NVLink contention at 70.59 Kbps with 4.78% error ³¹, and the earlier contention-based attack on multi-GPU interconnects showed the same class of vulnerability ³². Bandwidth scarcity: gradient compression remains the blunt instrument for bandwidth-constrained clusters, with 1-bit LAMB’s 4.6× communication reduction and 2.8× end-to-end speedup on Ethernet clusters ¹²⁰ and near-lossless compression at up to 67.1% volume reduction ¹²¹ as the representative results. Energy: the communication tax argument ⁹⁴ and energy-aware collective scheduling ⁵⁶ are the only direct measurements of interconnect energy in the corpus — a striking gap given the topic’s importance to datacenter power budgets.

In-network computing sits at the intersection: offloading aggregation into the network — host-driven in-network aggregation on RDMA ¹²², constrained in-network computing with congestion guarantees ¹²³, and the survey of in-network computing for distributed ML ¹²⁴ — promises to cut both latency and energy, at the price of programmable-switch complexity that production LLM clusters have not yet widely accepted.

Where the evidence disagrees

Lossless versus lossy RDMA. The position literature says losslessness is the problem ³¹¹, the reliability-engineering literature builds the lossy future ⁷¹⁰¹⁵, and the production evidence — Meta’s lossless RoCE fabric ¹ — shows losslessness working at scale with heavy engineering. The apparent conflict dissolves once separated by role: the critics diagnose the default lossless stack; the operators show what managed losslessness achieves; the lossy work is younger and unproven at LLM scale. Ultra Ethernet is the industry’s attempt to have both ³⁴. The remaining open question is whether UEC’s lossy semantics actually deliver at 100,000-GPU scale — no evidence exists yet.

Does congestion control matter for training? The CC literature assumes it does ¹⁸¹²¹⁴, but the one training-level measurement found policy choice barely moves end-to-end training throughput ⁴. Explanation: CC matters at the fabric’s saturation point — large clusters with incast from synchronized all-reduce — and the measurement was small-scale. Both camps are consistent with “CC matters at scale”; neither has measured it at the scale where it matters.

Scale-up versus scale-out. NVIDIA’s bet is that NVLink domains win ⁴⁵¹³¹; Meta’s field data shows Ethernet scale-out working for them ¹¹³²; UALink’s backers argue the open path ²⁸³⁶. The evidence cannot resolve this because the load-bearing sources are vendor-adjacent, the comparisons are on paper ²⁸, and the production datapoints are single operators. This is a live strategic disagreement, and the review treats it as such.

Is MoE all-to-all a fundamental bottleneck? DeepSeek’s production reports claim near-zero all-to-all overhead with co-designed kernels and overlap ⁶⁹, while the optimization literature treats all-to-all as the dominant cost ⁷¹⁷²⁴⁵. The reconciliation is hardware- and software-generation-dependent: DeepSeek’s result is on their fabric with bespoke engineering; the optimizations target the general case. The 52.4% latency cut reported by UBEP on production superpods ⁴⁵ suggests the general case still has a long way to go.

Disaggregate or not. Prefill/decode disaggregation wins for mixed workloads ¹⁰³¹⁰⁴¹⁰⁵, but long-context serving favors elastic sequence parallelism ¹⁰⁶, and the newest work says the dichotomy itself is wrong ¹⁰⁹. Population explains it: the disagreement is about workload — short-context chat favors disaggregation; long-context favors SP; agentic mixes break both.

Gaps and open questions

No independent measurements of the newest scale-up fabrics. NVLink 4/5, UALink, and GB200 NVL72-class domains are described by vendors, vendor-adjacent papers, and one hyperscaler ¹³²¹³¹²⁸. An independent, reproducible benchmark of NVLink-domain versus UALink versus Ethernet for the same training workloads would settle more than any other single experiment.
Lossy RDMA at LLM scale is untested. The lossy-fabric results ⁷⁸⁹ are prototypes and simulations; no published production deployment exists. Ultra Ethernet’s central claims ³⁴ rest on the same gap.
Interconnect energy is nearly unmeasured. Two sources ⁹⁴⁵⁶ are the entire corpus on tokens-per-watt effects of networking; the datacenter energy literature treats compute, not fabric, and the review found no field study of interconnect energy share in an LLM cluster.
KV-transfer economics at extreme scale. The KV-cache movement numbers come from three systems ¹⁰⁵¹⁰⁸¹⁰⁶; the paywalled NIKA analysis of optimal KV transfer could not be retrieved for this review, and no independent cost model exists.
CXL for LLM is emulation-only. Every CXL-for-inference result ⁹⁶⁹⁷⁹⁸⁹⁹ is simulation or emulation; production CXL memory pooling for LLM serving is unreported.
The collective stack lacks a standard benchmark. The xCCL survey documents fragmentation ³⁹; the field’s speedup claims (6.7× ¹³³, 8× ¹²⁵, 27% ²⁶, 52.4% ⁴⁵) are not comparable across papers because no common topology, workload, and NCCL-version baseline exists.
Optical OCS beyond TPU v4. The production evidence is a single operator’s two deployments ⁸⁷¹²⁶; everything else ⁸⁸⁸⁹⁹⁰ is testbed-scale.

Confidence and limitations

Confidence. Moderate overall, with a clear gradient. High confidence: the transport-layer diagnoses (lossless-fabric pathologies, CC limitations) are replicated across many independent groups and both fabric families ³²⁰⁴; the collective-communication wins are measured end-to-end in multiple systems ¹³³⁴¹⁵⁵; MoE all-to-all as a first-class bottleneck is agreed across vendors, hyperscalers, and academics ⁶⁵⁶⁷⁴⁵. Moderate-to-low confidence: everything about the 2025–2026 hardware generation — UALink, UEC, NVL72 — rests on position papers, simulations, or single-operator reports ²⁸³⁷³⁶.

Limitations of this review. 77 of 137 sources were read at abstract level (many ACM/IEEE papers are paywalled); numbers from those are hedged and stated as reported. The search was English-only, 2020-2026, and excluded gray literature — vendor whitepapers contain fabric details (e.g., NVLink 5 specifications) that peer-reviewed sources do not, so the newest-hardware picture is thinner than the industry’s own documentation. OpenAlex’s daily budget was exhausted mid-session and the search was carried by Crossref and the arXiv API thereafter; five selected sources (including one KV-transfer analysis and one collective-scheduling paper) were dropped as unretrievable. Semantic Scholar contributed nothing (429-limited from this IP all session). Finally, the corpus is concentrated in the 2023–2026 window — the field is young, and “consensus” statements here are consensus about a fast-moving target.

Jump to references ↓

Evidence table

key	design	sample	measure	finding	limitations	confidence	access	note
gangidi2024ethernet	systems paper	Meta's production RoCE (RDMA over Converged Ethernet) networks for distributed AI training	network reliability, scalability, operational efficiency	Presents the design, implementation, and operation of Meta's RoCE networks for distributed AI training; no quantitative results are reported in the abstract.	Abstract-only; operational details and numbers not available in the record.	moderate	abstract-only	Production evidence that RoCE/Ethernet is viable as the interconnect fabric for large-scale AI training.
batllecas2025scaling	benchmark	Production clusters running large-scale LLM training workloads; RoCE vs InfiniBand	scaling performance, throughput, training efficiency	RoCE achieves near-linear scaling performance comparable to InfiniBand when properly configured, per extensive experiments on production clusters (no specific numbers in abstract).	Abstract-only; no numeric speedups or cluster sizes reported.	moderate	abstract-only	Head-to-head benchmark of RoCE vs InfiniBand for LLM training, supporting cost-effective Ethernet-based interconnects.
hoefler2023datacent	survey	RoCE/RDMA design points across hyperscale datacenters; analysis uses 800 Gb/s links, 600 ns hop latency, 9 kB frames	headroom buffer requirements, packet rates, header overhead, retransmission bandwidth loss	RoCEv2 headers total 66 B vs 20 B for InfiniBand, cutting the max packet rate on an 800 Gb/s link from 12.5 to 1.4 Gpps for 8-byte messages, and the paper predicts modernized Ethernet will replace TCP and RoCE within a decade.	Position/vision paper by industry authors, not an empirical evaluation; quantitative examples are illustrative.	high	full-text	Foundational critique of RoCE/PFC (headroom buffers, victim flows, go-back-N, ECN) that motivates Ultra Ethernet for AI datacenters.
khan2022impact	simulation	ASTRA-SIM + NS3 simulation of DLRM training on a 2-level Clos platform: 8 GPUs/server over 200 GB/s NVLink, 200 Gbps NIC-to-TOR links, up to 16 racks	end-to-end training iteration time, exposed communication time, PFC PAUSE frames, switch buffer occupancy	State-of-the-art RoCE congestion control schemes (DCQCN, DCTCP, TIMELY, HPCC) have little impact on end-to-end training performance versus baseline PFC on distributed training platforms.	Simulation only (ASTRA-SIM/NS3); single workload (DLRM) and single-tenant assumption; no hardware testbed.	high	full-text	Shows general-datacenter congestion control does not transfer to collective-based (All-Reduce/All-To-All) training networks, guiding interconnect CC design.
he2025adaptive	systems paper	Testbed experiments on RDMA data center networks (lossless, PFC-based)	flow completion time (FCT), convergence speed, transmission rate	FACC, a switch-driven RDMA congestion control algorithm, improves convergence speed while achieving at most 86.6% lower flow completion time compared with state-of-the-art approaches.	Abstract-only; testbed scale and hardware not stated.	moderate	abstract-only	RDMA congestion control optimization for latency-sensitive AI applications on lossless fabrics.
li2025revisiti	systems paper	P4 switch prototype (DCP-Switch) and FPGA RNIC prototype (DCP-RNIC) on high-speed lossy fabrics	throughput and performance relative to SOTA lossless and lossy RDMA solutions	DCP, a PFC-independent transport co-designing switches and RNICs, achieves 1.6x and 2.1x performance improvements compared to state-of-the-art lossless and lossy RDMA solutions, respectively.	Prototype hardware (P4/FPGA), not production ASICs; abstract-only details.	moderate	abstract-only	Enables efficient RDMA over lossy (non-PFC) Ethernet fabrics, relevant to RoCE scalability for LLM training.
khalilov2025software	systems paper	SDR-RDMA on NVIDIA BlueField-3 SuperNIC (DPA offload) plus simulation; measured 350 km inter-DC link (100 Gbit/s, drop rates 1e-4 to 1e-1) and links up to 3.2 Tbit/s	average and 99.9th-percentile RDMA Write completion time, message completion time, packet drop rate	Guided selection between Selective Repeat and Erasure Coding reliability improves average and 99.9th-percentile RDMA Write completion time by up to 5x and 12x respectively on long-haul links, with the SDR backend sustaining line-rate packet processing up to 3.2 Tbit/s.	Requires BlueField-3 DPA offload; inter-datacenter (lossy, ms-RTT) focus rather than intra-cluster; partially simulation-based.	high	full-text	Software-defined reliability stack for planetary-scale RDMA, key for multi-datacenter LLM training collectives.
lin2025shift	systems paper	rdma-core userspace implementation evaluated with PyTorch distributed training; cites Alibaba (15.8% of training failures network-related), Azure (8.3% of InfiniBand failures), Tencent (15% NIC, 30% switch/fiber) production data	overhead during normal operation, failure masking capability, training continuity	SHIFT proves an RDMA Failover Trilemma (exactly-once execution, receiver-NIC opacity, and zero-copy cannot coexist) and, exploiting idempotent bulk transfers in NCCL/NVSHMEM, masks fatal NIC failures and link anomalies with negligible overhead, avoiding costly training restarts.	Does not support NCCL LL/LL128 or atomic-notification protocols; backup-NIC path is constrained by PCIe bandwidth and co-located traffic.	high	full-text	Extends RDMA fault tolerance to cross-NIC level and quantifies network failures' share in large-scale LLM training.
hua2026semantic	systems paper	Testbed with Mellanox ConnectX-5 RNICs (4 hosts, 100 Gbps switch) plus ns-3 simulations of large-scale collective communication	99th-percentile flow completion time, QP scalability, RNIC throughput, PCIe bandwidth overhead	CSA-UD, a communication-semantic-aware unreliable-datagram loss recovery scheme, achieves over 30% lower 99th-percentile flow completion time than UD counterparts and better scalability than RC, whose throughput degrades once concurrent QPs exceed roughly 512 due to PCIe metadata-fetch penalties.	Software-based recovery runs on the host CPU; requires UD mode and per-flow bitmap state; RNIC cache limits (~5,000 QPs) motivate design.	high	full-text	Addresses QP scalability and tail latency of RDMA collectives (All-Reduce/All-to-All) for trillion-parameter training.
yang2025adaptive	systems paper	RDMA data center networks sensitive to packet loss (Go-Back-N retransmission); FEC-based scheme	packet loss recovery effectiveness, throughput (not reported)	Proposes Forward Error Correction with adaptive redundancy to recover lost packets and mitigate Go-Back-N retransmission costs; the abstract is truncated and reports no quantitative results.	Abstract truncated; no numbers, hardware, or evaluation details available.	low	abstract-only	FEC-based alternative to lossless fabrics for RDMA, relevant to RoCE reliability design.
snyder2023congesti	field study	RDMA deployed in shared cloud environments; congestion control enforcement and algorithms	short flow completion time, attacker bandwidth, congestion severity	Performance hacking attacks exploiting RDMA congestion control can give attackers extra bandwidth and increase short flow completion times by several orders of magnitude, exposing a fundamental tradeoff between short-flow latency and performance isolation.	Abstract-only; no experimental setup or numbers reported.	moderate	abstract-only	Security/isolation angle on RDMA congestion control for shared AI clouds.
wan2025revisiti	systems paper	Commodity servers and RNICs; RDMA intra-host network experiments	network throughput and latency under intra-host congestion	RHCC, an RDMA intra-host congestion control scheme with sub-RTT-granularity avoidance and proactive RNIC traffic adjustment, can increase throughput and decrease latency by up to 2x and 1.4x, respectively.	Abstract-only; intra-host scope; inter-host CC interaction only briefly covered.	moderate	abstract-only	Highlights intra-host congestion as an emerging bottleneck as RNIC speeds outpace host interconnects.
shan2026headroom	systems paper	PFC-enabled commodity Ethernet switches; dynamic/shared headroom allocation (DSH)	flow completion time, buffer utilization, PFC message frequency	DSH, which dynamically allocates and shares per-queue headroom under port-level flow control, reduces flow completion time by up to ~78.8% in extensive experiments.	Abstract-only; switch model and scale not stated.	moderate	abstract-only	Headroom buffer inefficiency in lossless Ethernet, a core cost of PFC-based RoCE deployments.
liu2026accurate	systems paper	DPDK prototype of FAR rate-control protocol on datacenter networks	tail flow completion time (FCT), bandwidth estimation accuracy, queue oscillation	FAR, which couples bandwidth-estimation-based congestion control with end-to-end flow pausing, reduces tail FCT by up to 67% compared with state-of-the-art designs.	DPDK software prototype; abstract-only details.	moderate	abstract-only	Combines flow pausing and congestion control for low-latency datacenter (incl. RDMA) networks.
planeta2025converge	systems paper	RDMA networks in cloud/HPC settings; CoRD removes kernel bypass from the RDMA dataplane	end-to-end application performance, OS-level control overhead	CoRD shows kernel bypass is not mandatory for high-performance networking and can improve end-to-end application performance by up to 10% in one use case while enabling OS control of the RDMA dataplane.	Abstract-only; single demonstrated use case.	moderate	abstract-only	Challenges the kernel-bypass assumption of RDMA stacks, relevant to cloud-hosted RDMA AI training.
li2023gleam	systems paper	Fully functional Gleam prototype: switch with extended multicast forwarding table + 4 commodity servers with unmodified RNICs; large-scale simulations	multicast communication time, data replication throughput, HPL communication time	Gleam, which re-purposes RDMA RC logic with switch coordination for in-fabric multicast, achieves 2.9x lower communication time for the HPL benchmark and 2.7x higher data replication throughput (up to 2.2x multicast speedup on testbed).	Requires switch modifications (layer-4 states in forwarding table); evaluated on HPC/storage multicast patterns rather than LLM workloads.	high	full-text	Brings reliable multicast to RDMA RC transport, a building block for group communication in training frameworks.
tian2021multi	systems paper	Maestro software middleware deployed on existing RDMA NICs for distributed deep learning (DDL) applications	multi-path utilization, CPU overhead, DDL application performance	Maestro, a software-defined multi-path RDMA solution, fully utilizes multiple paths with negligible CPU overhead and enhances DDL application performance (no numeric results in abstract).	Abstract-only; no numbers; user-space middleware overhead not quantified in record.	moderate	abstract-only	Software multi-path load balancing for RDMA, addressing the single-path limitation of RNIC hardware.
luo2024flowstar	systems paper	InfiniBand networks (per-flow CBFC on switches); evaluation across different workloads	average and tail message completion time	FlowStar, a per-flow accurate InfiniBand congestion control combining optimized per-flow CBFC with rate adjustment, improves average and tail message completion time under different workloads (no numeric values reported).	Abstract-only; no numbers; requires per-flow state on switches.	moderate	abstract-only	InfiniBand congestion control addressing victim flows and slow-convergence congestion spreading.
cascajo2023monitori	field study	Real InfiniBand-based cluster combining LIMITLESS monitoring with OpenSM control software	number of wrong reactions of the InfiniBand congestion-control mechanism	Dynamically reconfiguring InfiniBand congestion-control parameters based on monitoring data significantly reduces the number of wrong reactions to the congestion-control mechanism (no numbers reported).	Abstract-only; no quantitative results; parameter tuning approach is cluster-specific.	moderate	abstract-only	Operational tuning of InfiniBand congestion control parameters in real HPC clusters.
maglionem2020path2sl	systems paper	InfiniBand fat-tree network topologies; Path2SL Virtual Lane queuing scheme	head-of-line (HoL) blocking reduction, network performance under congestion	Path2SL optimizes the use of InfiniBand Virtual Lanes to reduce HoL blocking in fat-tree topologies (no numeric results in abstract).	Abstract-only; no numbers; evaluation context (simulation vs testbed) not stated.	moderate	abstract-only	InfiniBand queuing/Virtual Lane mechanisms for congestion mitigation in HPC interconnects.
katebzade2020evaluati	benchmark	Rack-scale InfiniBand cluster benchmarked with RPerf in isolated and mixed latency/bandwidth traffic scenarios	switch latency, bandwidth, and latency-bandwidth tradeoff under mixed traffic	The evaluated InfiniBand switch can provide either low latency or high bandwidth, but not both simultaneously in mixed-traffic scenarios, and none of the evaluated improvement options is ideal.	Single switch model; measurement tool (RPerf) without hardware support; abstract-only.	moderate	abstract-only	Measurement methodology (RPerf) and evidence on InfiniBand switch behavior under mixed workloads.
luo2024alarm	simulation	Simulated HPC networks with InfiniBand-style deterministic routing; one-way-delay-guided adaptive routing	average and p99 flow completion time (FCT) slowdown	Alarm, a probabilistic adaptive routing system using one-way delay metrics with per-flowlet decisions, achieves 2.0x and 1.7x better average and p99 FCT slowdown than existing adaptive routing.	Simulation-based evaluation; abstract-only details.	moderate	abstract-only	Adaptive routing to balance traffic in InfiniBand-dominated HPC/AI clusters.
wei2023nvlink	survey	NVIDIA Grace-Hopper and Grace Superchip systems using NVLink-C2C	link bandwidth, cache-coherent memory access	NVLink-C2C provides a 900 GB/s link between Grace and Hopper (or two Grace chips) with a unified, cache-coherent memory address space combining system and HBM GPU memories.	Vendor product overview; abstract-only; no independent measurements.	moderate	abstract-only	Documents NVLink-C2C as the high-bandwidth CPU-GPU interconnect enabling unified memory for AI/HPC.
ishii2022nvlink	survey	Historical overview of NVLink generations	not reported	The abstract contains only the title 'Brief History of NVLink'; no content or numbers are available in the record.	Abstract effectively empty; cannot extract findings.	low	abstract-only	Historical context source for NVLink; minimal extractable evidence.
li2020evaluati	benchmark	Six platforms: P100-DGX-1, V100-DGX-1, DGX-2, OLCF SummitDev and Summit, SLI system with 2x RTX-2080; interconnects PCIe, NVLink-V1/V2, NVLink-SLI, NVSwitch	GPU communication bandwidth, NUMA effects, multi-GPU application performance	The evaluation identifies four new types of GPU communication network NUMA effects (three triggered by NVLink topology/connectivity/routing and one by a PCIe chipset design issue), showing GPU choice can considerably impact communication efficiency (no numeric bandwidths in abstract).	Older hardware generations (P100/V100/RTX-2080 era); abstract-only.	moderate	abstract-only	Empirical characterization of intra-node GPU interconnects (PCIe/NVLink/NVSwitch) foundational to multi-GPU training.
shen2025flexlink	systems paper	8-GPU H800 server (NVLink 400 GB/s, PCIe Gen5 x16 ~128 GB/s, 800 Gb/s NICs); MoE training and long-sequence inference workloads	AllReduce/AllGather bandwidth, communication time share, load distribution	FlexLink aggregates NVLink, PCIe, and RDMA NICs with two-stage adaptive load balancing, improving AllReduce and AllGather bandwidth by up to 26% and 27% over NCCL by offloading 2-22% of traffic to previously idle links; MoE communication can consume up to 43.6% of forward-pass time and Flash Communication overhead up to 65.9%.	H800-specific (reduced NVLink); PCIe path contention caps gains; gains depend on idle secondary-link bandwidth (14-33% across surveyed platforms).	high	full-text	Directly addresses intra-node interconnect utilization (NVLink+PCIe+RDMA aggregation) for LLM training/inference collectives.
werner2025memory	benchmark	NVIDIA Grace-Hopper system; CPU accessing GPU memory via NVLink-C2C for memory expansion	memory access throughput and latency; database index operation performance	CPU access to GPU memory over NVLink-C2C achieves throughput of up to 168 GB/s with access latencies between about 800 ns and 1000 ns on the Grace-Hopper system.	Single system generation; microbenchmark scope; abstract-only.	moderate	abstract-only	Quantifies NVLink-C2C viability for memory expansion, informing memory-disaggregation interconnect choices.
vemuri2025centric	theoretical	Analytical models of latency- and bandwidth-sensitive transformer collectives over UALink 1.0 (up to 1024 accelerators/pod, <1 us request-to-response) vs NVLink 3-5 (NVLink 5 ~1.8 TB/s per-GPU)	step-time contribution of collectives under flat and hierarchical topologies	A flat UALink pod can reduce the step-time contribution of small-message collectives at scales up to 1024 GPUs, while NVLink 5 maintains an advantage in bandwidth-dominated regimes due to aggregate per-GPU bandwidth near 1.8 TB/s and large non-blocking pods.	Analytical/figure-generation models, not empirical measurements; abstract-only.	moderate	abstract-only	Compares open UALink vs proprietary NVLink for AI-training collectives, informing interconnect standardization.
choi2022multi	benchmark	Multi-GPU servers with up to 8 A100 GPUs connected by NVLink 3.0 links through NVSwitch	matrix-matrix multiplication performance	Analyzes the performance of matrix-matrix multiplication on NVLink 3.0/NVSwitch-connected 8-A100 servers; the abstract is truncated and reports no numeric results.	Abstract truncated; no numbers available.	low	abstract-only	Intra-node NVLink/NVSwitch performance analysis for multi-GPU compute kernels.
luo2026c2cserve	systems paper	GH200 Superchips with MIG partitioning; three-week Alibaba production trace of 89 models; Llama 3 series workloads	cold-start latency, TTFT/TPOT attainment, throughput under shared NVLink-C2C contention	C2CServe keeps model weights in CPU memory streamed over NVLink-C2C (~450 GB/s per direction, ~7x PCIe 5.0 x16) to MIG instances, reducing cold-start latency by up to 7.1x for dense and 4.6x for MoE models versus state-of-the-art serverless systems while maintaining over 95% TTFT/TPOT attainment under C2C contention.	GH200-specific; C2C bandwidth is shared across MIG instances (interference gap widens 28% to 42% as co-located model sizes grow 5 GB to 44 GB); requires Superchip hardware.	high	full-text	Shows NVLink-C2C bandwidth reshaping LLM inference/serving architecture, an interconnect-centric serving system.
zhang2025nvbleed	case study	DGX-1 (8x Tesla P100, NVLink-V1) and GCP 8x Tesla V100 (NVLink-V2) cloud instances; 18 HPC/DL applications; 50 Blender characters	covert channel bandwidth and error rate; side-channel F1/accuracy scores	NVBleed exploits NVLink contention timing and performance counters to build covert channels achieving 70.59 Kbps with 4.78% error rate, application fingerprinting with F1 up to 97.78%, Blender character identification with 91.56%, and cross-VM attacks on GCP exceeding 88% F1.	Tested on NVLink-V1/V2 only (not V3/NVSwitch); performance-counter leakage can be restricted by a NVIDIA driver patch; contention-based channel needs co-location.	high	full-text	Security analysis of the NVLink interconnect in multi-GPU and cloud AI systems, documenting leakage via contention and counters.
zhang2024beyond	case study	Multi-GPU systems sharing the NVLink interconnect (CPU-GPU and GPU-GPU)	covert channel bandwidth and error rate; application fingerprinting accuracy	A covert channel attack across two GPUs achieves 45.5 kbps with a low error rate, and NVLink congestion monitoring enables application fingerprinting without special permissions.	Abstract-only; fewer experimental details than the companion NVBleed study.	moderate	abstract-only	Demonstrates NVLink congestion leakage as a side channel in multi-GPU systems.
singhania2025understa	benchmark	Perlmutter (A100, intra-node NVLink, inter-node HPE Slingshot-11) and Vista (GH200, InfiniBand); Llama 3.1 70B/405B with vLLM, SGLang, and YALIS engines; 4 to 128 GPUs	end-to-end batch latency, all-reduce latency, strong-scaling efficiency	NVRAR, a hierarchical NVSHMEM-based recursive-doubling all-reduce, achieves up to 1.9x-3.6x lower latency than NCCL for 128 KB-2 MB messages on Slingshot/InfiniBand and up to 1.72x lower end-to-end batch latency for Llama 3.1 405B tensor-parallel decode-heavy inference, while NCCL inter-node all-reduce is 1.5-2x slower than MPI for 512 KB-1 MB messages.	Preliminary work (under review); decode-heavy focus; YALIS is a research prototype; scaling experiments limited to two HPC systems.	high	full-text	Shows all-reduce/inter-node interconnect (Slingshot, InfiniBand) as the key bottleneck in multi-node LLM inference.
hoefler2025ultra	survey	Ultra Ethernet (UE) 1.0 specification (562 pages) authored by UEC members; backend networks at 400+ Gbps over 10-150 m links	design parameters: connectionless transport (UET), ECMP packet spraying via entropy values, PDC reliability, NSCC/RCCC congestion control, TSS encryption, CBFC/LLR link extensions	UE 1.0 replaces connection-oriented, lossless (PFC-based) RDMA/RoCE semantics with a connectionless, packet-sprayed transport designed for millions of endpoints, with ECMP hash-conflict probability of 25% (same group) / 6.25% (different groups) motivating per-packet entropy values.	Specification overview by the authors; no performance measurements or benchmarks; vendor differentiation leaves many mechanisms optional.	high	full-text	The Ethernet ecosystem's answer to InfiniBand for AI/HPC: defines the modernized high-performance Ethernet transport.
metz2024empoweri	survey	Ultra Ethernet Consortium (UEC) and its goals for AI/HPC workloads on Ethernet	qualitative: physical-layer tradeoffs (power, temperature, distance), transport-layer reliability and latency	Discusses the UEC's push to refine the Ethernet stack, particularly the transport layer, to meet AI workloads' scale, latency, and reliability demands; no quantitative results are reported.	Position/discussion paper; abstract-only; no numbers.	moderate	abstract-only	Motivation and consortium context for Ultra Ethernet as the AI interconnect standard.
arsid2026ualink	survey	Open-standard interconnect technologies for AI/HPC datacenters: UALink, Ultra Ethernet, and PCIe/CXL (literature survey)	Qualitative bandwidth/latency capabilities and feature comparison	Survey reports UALink targets multi-terabit throughput at sub-microsecond latency in vendor-neutral topologies, Ultra Ethernet adds deterministic forwarding, advanced congestion control, and hardware-accelerated collectives to commodity Ethernet, and CXL enables cache-coherent memory tiering with order-of-magnitude performance gains over software-managed tiering (specific numeric benchmarks not reported).	Secondary source with qualitative claims; no primary experiments or measured benchmarks.	moderate	abstract-only	Positions UALink/Ultra Ethernet/CXL as the emerging hierarchical interconnect stack for scale-out AI, directly relevant to RDMA/Ethernet/RoCE and scale-up fabrics for LLM training clusters.
alladi2026impact	simulation	Simulated reconfigurable NIC prioritizing Ultra Ethernet Transport (UET) egress traffic at 200 Gbps with co-located non-UET tenants	Added average latency for non-UET tenants (percent and ns)	Prioritizing UET egress traffic increases average latency for co-located non-UET tenants by less than 2% (under 2 ns absolute at 200 Gbps), a threshold the authors argue is inconsequential.	Simulation-based analysis; single NIC-modification scenario; not validated on real FPGA-based UET NIC implementations.	moderate	abstract-only	Supports incremental adoption of Ultra Ethernet (RoCE-class) transport sharing NICs with legacy tenants in hyperscaler clouds for AI/HPC traffic.
hu2025demystif	systems paper	NCCL 2.19.1 source/design analysis plus ATLAHS application-trace-driven network simulator for AI training workloads	Qualitative analysis of protocols (Simple, LL, LL128), transports (NVLink, PCIe, IB verbs/GPUDirect RDMA), ring/tree algorithms	Provides an in-depth analysis of NCCL's communication protocols, channel orchestration, intra/inter-node data movement, and ring/tree collectives, and builds ATLAHS to accurately reproduce NCCL communication patterns at scale (no headline performance numbers reported).	Analysis based on NCCL 2.19.1 so implementation details may evolve; qualitative rather than quantitative benchmarking.	high	full-text	Foundational reference demystifying NCCL internals (protocol variants, GPUDirect RDMA transports, ring/tree algorithms) that underpin large-scale LLM training interconnect stacks.
weingram2023survey	survey	Industry-led collective communication libraries (xCCL): NCCL, oneCCL, RCCL, MSCCL, ACCL, Gloo	Design features, use cases, and performance comparison using NCCL Tests and PARAM benchmarks	Surveys design features and industry use cases of six xCCL libraries and compares their performance with the NCCL Tests and PARAM industry benchmarks (specific numeric results not reported in abstract).	Survey-level overview; benchmark numbers not detailed in abstract; library landscape evolves rapidly.	moderate	abstract-only	Maps the collective communication library landscape (NCCL/RCCL/MSCCL/ACCL/Gloo) relevant to interconnect-aware optimization for deep learning workloads.
cowan2023mscclang	systems paper	MSCCL++ framework/language for custom collective algorithms on large multi-GPU systems	Qualitative: ability to implement topology- and pattern-specific collectives	Proposes MSCCL++ for implementing correct and efficient custom collective algorithms optimized for both network topology and application-specific communication patterns to alleviate collective bottlenecks in multi-GPU training/serving (no numeric results reported in abstract).	Abstract-only; no performance numbers available locally; requires custom-algorithm implementation expertise.	moderate	abstract-only	Enables topology-aware custom collectives (e.g., for NVLink/InfiniBand topologies), an approach later built upon by synthesis-based schedulers such as TACCL and Canvas.
hei2025canvas	systems paper	Collective-communication scheduling synthesis evaluated on topologies up to 512 GPUs; baselines TACCL and TE-CCL	Bandwidth speedup and schedule synthesis time	Canvas achieves up to 1.98x bandwidth speedup over TACCL and 3.56x over TE-CCL, and synthesizes algorithms for 512-GPU topologies within 1.77 hours whereas TACCL fails to produce results within 24 hours.	Evaluated on imbalanced topologies; hierarchical decomposition may trade optimality; cross-micro-batch pipelining adds complexity.	high	abstract-only	Advances scalable, near-optimal topology-aware collective scheduling to overcome vendor-library bandwidth bottlenecks in large GPU clusters used for LLM training.
cao2024scheduli	systems paper	96-GPU testbed experiments plus large-scale production trace-based simulation of multitenant deep-learning-training jobs	GPU computation utilization (%)	Crux, a GPU-intensity-aware communication scheduler, improves GPU computation utilization by 8.3%-14.8% on a 96-GPU testbed and by up to 23% in production-trace simulation versus Sincronia, TACCL, and CASSINI.	Multitenant cloud setting; communication-contention focus; NP-complete problem addressed via heuristic theorem-based approach.	high	abstract-only	Shows communication contention among co-located DLT jobs wastes GPU compute and that prioritizing high-intensity flows mitigates it in shared training clusters.
huang2023gzccl	systems paper	Up to 512 NVIDIA A100 GPUs on HPE Slingshot 10 (100 Gbps) interconnect; real-world datasets and image-stacking application	Collective speedup vs NCCL/Cray MPI; reconstructed data quality	gZCCL-accelerated Allreduce and Scatter outperform NCCL by up to 4.5x and Cray MPI by up to 28.7x (gZ-Allreduce ring up to 20.2x over Cray MPI/NCCL), with a 1.69x gain in image stacking while preserving high data quality under a 1E-4 error bound.	Lossy compression accuracy depends on error bounds; compression/decompression still occupies substantial runtime share; evaluated on Slingshot 10-class fabrics.	high	full-text	Demonstrates accuracy-aware lossy compression inside GPU-aware collectives as a way to stretch limited inter-node bandwidth (100 Gbps-class) in large-scale training.
li2024optimizi	field study	Operational GPU-centric network managing over 10K heterogeneous GPU cards (production deployment)	Qualitative: collective communication latency, congestion, and utilization of GPU and switch network bandwidth	TCCL jointly optimizes topology-aware collective communication at hosts and centralized routing management in multi-path networks, fully utilizing GPU and switch network bandwidth in parallel and reducing collective delay via short cross-node paths (specific numbers not reported in abstract).	Practice/experience report; quantitative results not given in abstract; single-operator context.	moderate	abstract-only	Field evidence that GPU-centric networks (>10K GPUs) require co-design of collective communication and centralized routing to avoid congestion and jitter in LLM training.
liu2026architec	systems paper	UBEP evaluated on up to 256 NPU dies from a production Huawei CM384 superpod (Unified Bus fabric, ~400 GB/s, hundreds-of-ns latency); MoE inference	All-to-All latency reduction and MoE inference Time Per Output Token (TPOT)	UBEP reduces All-to-All latency by up to 52.4% and MoE inference TPOT by up to 11.1% versus the baseline CANN EP, where all-to-all communication consumes roughly 47% of total execution time.	Evaluated on Huawei CANN/NPU stack only; analysis specific to superpod scale-up fabrics (NVL72/576, CloudMatrix384) rather than traditional IB/RoCE clusters.	high	full-text	Quantifies how BSP serialization, synchronization overhead, and distance-agnostic scheduling waste high-bandwidth scale-up fabrics (UB/NVLink-class) in MoE all-to-all traffic.
goldman2026unified	systems paper	NCCL EP library evaluated on an H100-based cluster across 1-8 node configurations; vLLM and Megatron-LM integration; NVLink intra-node + RDMA (GIN) inter-node	Low-latency kernel latency (decode, 1-128 tokens), high-throughput kernel throughput (4096+ tokens), end-to-end vLLM results	NCCL EP provides unified ncclEpDispatch/ncclEpCombine primitives on NCCL's Device API with LL mode (1-128 tokens) and HT mode (4096+ tokens), achieving competitive LL kernel performance versus DeepEP and end-to-end results with vLLM (specific latency/throughput numbers not reported in read portions).	Specific numeric results not captured in the portions read; evaluation limited to H100 clusters; requires NCCL 2.28+ Device API/GIN support.	moderate	full-text	Brings MoE dispatch/combine (fine-grained all-to-all over RDMA+NVLink) natively into NCCL, unifying device-initiated expert-parallel communication for training and inference.
zheng2026ncclbpf	systems paper	8x NVIDIA B300 GPUs connected via NVLink; NCCL 2.29.7 with bpftime userspace eBPF runtime	Tuner-decision overhead (ns), AllReduce throughput, policy hot-reload downtime	NCCLbpf adds 80-130 ns overhead per tuner decision (less than 0.03% of collective latency), hot-reloads policies atomically in 1.07 us with zero dropped calls across 400,000 invocations, and improves 8-GPU AllReduce throughput by up to 27% over NCCL defaults in the 4-128 MiB range.	Single 8-GPU NVLink-node evaluation; selects among NCCL built-in algorithms only (no custom algorithms); userspace eBPF verification scope.	high	full-text	Demonstrates verified, composable, hot-reloadable policy extension (eBPF-style) for NCCL's plugin interface, improving collective throughput without modifying NCCL source.
jia2026inspecto	case study	Nemotron-4 LLM pretraining (340B model) on NVIDIA H100 clusters with 16 to 2,048 GPUs; NVLink/PCIe intra-node and InfiniBand/RoCE inter-node	NCCL Inspector metrics: algorithmic/hardware bus bandwidth, latency, message size per collective type	NIXT converts NCCL Inspector profiling into accessible analysis, attributing straggler anomalies and showing how communication phases change with ML parallelism and GPU scale during Nemotron-4 pretraining up to 2,048 GPUs, where collective communication commonly consumes 20-40% of training time.	Case study on one model family and cluster; observability tooling rather than an optimization itself.	high	full-text	Provides production-scale observability into collective communication (bandwidth, stragglers, phase attribution) needed to diagnose interconnect bottlenecks in LLM pretraining.
shen2026every	systems paper	NCCL device-side low-latency collective kernels on GPU scale-up networks; Llama-3.1-70B tensor-parallel inference; cuSOLVERMp; microbenchmarks vs speed-of-light bound	Small-message AllReduce latency (us), inter-token latency (ITL), cost per 1M output tokens	New symmetric NCCL kernels reduce small-message AllReduce latency from 11.0 us (NCCL ring) to 2.37 us, within 7% of the hardware speed-of-light lower bound, yielding an 8.7% ITL reduction for Llama-3.1-70B inference and about 0.9% cost saving per microsecond removed from AllReduce latency.	Latency-focused optimizations for small/medium messages; scale-up (single-node NVLink-class) networks; bandwidth-bound large messages benefit less.	high	full-text	Shows latency, not just bandwidth, dominates small collectives on decode-heavy LLM inference critical paths, and quantifies microsecond-level gains achievable on scale-up interconnects.
singh2025scalable	systems paper	PCCL evaluated on Frontier (2,048 AMD MI250X GCDs) and Perlmutter (NVIDIA A100); DeepSpeed ZeRO-3 and DDP training workloads	All-gather/reduce-scatter/all-reduce speedup; end-to-end training throughput speedup	PCCL achieves up to 168x faster reduce-scatter, 33x faster all-gather, and 10x faster all-reduce than RCCL on 2,048 Frontier GCDs (up to 5.7x over NCCL on Perlmutter), translating to up to 4.9x DeepSpeed ZeRO-3 and 2.4x DDP training speedups.	Optimized for large messages (>10 MB); hierarchical learning-based algorithm selection may not generalize to all topologies; vendor-specific tuning.	high	full-text	Quantifies large headroom in vendor collective libraries (RCCL/NCCL) at supercomputer scale, informing interconnect-aware collective design for large-buffer DL workloads.
zhang2025reliable	systems paper	Production GPU training clusters (largest deployment 24K GPUs); 8-GPU Hopper + 8 ConnectX-7 RNIC testbed with 400 Gbps 1:1 two-tier CLOS	Training throughput, inter-node small-message latency, GPU time wastage under RNIC failures	VCCL improves training throughput by up to 5.28% over NCCL, reduces inter-node small-message latency by 18.9% on average, and reduces GPU time wastage by nearly 90% during NIC port failures via a primary-backup QP mechanism.	Single-company deployment experience; healthy-path gains are percent-level; requires multi-NIC servers and RDMA monitoring support.	high	full-text	Production evidence that SM competition, RNIC port failures, and transient RDMA anomalies waste GPU resources, and that SM-free P2P plus fault-tolerant QPs recover most of the loss.
zhang2023grained	systems paper	64-GPU cluster with 10Gb/s Ethernet and 100Gb/s InfiniBand interconnects; five popular DNN models	Training speedup versus state-of-the-art all-reduce scheduling	DeAR decouples all-reduce into two continuous operations overlapped with both backpropagation and feed-forward, achieving up to 83% training speedup on 10Gb/s Ethernet and 15% on 100Gb/s InfiniBand, reaching 72.3-99.2% of the maximum possible speedup.	Evaluated at 64-GPU scale; gains depend on interconnect latency/bandwidth regime; scheduling complexity grows with model size.	high	full-text	Shows fine-grained overlap of decoupled all-reduce with both backward and forward passes, with larger gains on low-bandwidth/high-latency Ethernet than on InfiniBand.
wang2023extremel	systems paper	GPT-3-like models (10-138B parameters) trained on up to 384 GPUs; low-bandwidth clusters with cross-node links often below 100 Gbps Ethernet	Communication volume reduction, throughput speedup, sustained peak throughput	ZeRO++ reduces ZeRO communication volume 4x (block-quantized all-gather cutting weight gathering 50%, hierarchical remapping eliminating inter-node backward all-gather, all-to-all quantized gradient reduction cutting gradients 75%), enabling up to 2.16x better throughput at 384 GPUs, over 45% sustained peak throughput, and up to 2.4x end-to-end speedup over ZeRO.	Lossy quantization needs convergence checks; hierarchical partitioning trades memory for communication; evaluated up to 384 GPUs.	high	full-text	Communication-volume reduction techniques that make ZeRO-3 viable on bandwidth-limited Ethernet clusters, a key regime for LLM training outside InfiniBand/NVLink datacenters.
park2026fully	systems paper	Commodity (bandwidth-limited) GPU cluster; ZeRO-3 fully sharded data parallel training and parameter-efficient fine-tuning (PEFT)	Inter-node all-gather traffic reduction and training throughput	FCDP caches forward-pass parameters in host memory for fast intra-node reuse, reducing inter-node all-gather by 50% (over 99% for PEFT) and achieving up to 100x higher throughput than ZeRO-3 and 51x over ZeRO++ while keeping ZeRO-3's minimal GPU memory footprint.	Assumes host-memory bandwidth beats inter-node links; PEFT-specific gains rely on selective communication; commodity-cluster focus.	high	abstract-only	Reframes host memory as a fast caching layer to bypass slow inter-node interconnects, a practical trick for LLM training on non-NVLink/InfiniBand commodity fabrics.
xu2026lagom	systems paper	High- and low-bandwidth GPU clusters; diverse models and parallelization strategies; baselines NCCL and AutoCCL	Training speedup from communication-computation overlap tuning	Lagom co-tunes communication parameters with a unified cost model and priority-based search (reducing complexity from exponential to linear), achieving 1.07-1.33x speedup over NCCL and 1.03-1.27x over AutoCCL, where mis-tuned communication contention degrades bottlenecked computation by up to 35%.	Tuning scope limited to NCCL-style communication parameters; search heuristic may miss global optimum; per-cluster calibration needed.	high	full-text	Shows communication tuning must also target computation-bottlenecked regimes, informing overlap strategies on both RDMA and NVLink fabrics in LLM training.
jia2024energy	systems paper	Characterization of NCCL collective operations plus PCCL DVFS-based power management on GPU clusters; end-to-end LLM training	Energy consumption of collective communication and end-to-end training energy	PCCL identifies the optimal GPU frequency per collective call and reduces collective communication energy by roughly 27% and end-to-end LLM training energy by 17.3% with negligible throughput impact, where collectives often account for more than half of training time.	DVFS benefits depend on workload and GPU model; power measurement granularity; interconnect specifics (NVLink/InfiniBand) not detailed in abstract.	moderate	abstract-only	Adds an energy dimension to interconnect/collective design: communication kernels dominate training time and offer DVFS-based power savings with minimal performance cost.
jin2024software	systems paper	SDCC software-defined collective communication framework; PyTorch and TensorFlow engines; Linux TCP and kernel-bypass transports	Qualitative: flexibility, generality, and performance via dataflow abstraction	SDCC decouples control plane from data plane using a unified dataflow graph, enabling users to express new collective algorithms, integrate multiple compute engines and network transports (TCP and kernel bypass), and exploit parallelism, demonstrated in four use cases (specific numbers not reported in abstract).	Abstract-only; no quantitative results available locally; general framework may add overhead versus hand-tuned vendor libraries.	moderate	abstract-only	Software-defined, transport-agnostic collectives (including kernel-bypass/RDMA-class transports) as an alternative to tightly coupled vendor communication stacks.
li2024understa	benchmark	Systematic empirical characterization of distributed training communication across determinant factors (parallelism, models, clusters)	Communication pattern features (e.g., predictability) and communication overhead factors	Provides a systematic analysis of communication patterns and overhead determinants and develops a validated analytical formulation to estimate communication overhead for models exhibiting predictability (specific numeric findings not reported in abstract).	Abstract-only; characterization may be tied to specific frameworks and cluster configurations.	moderate	abstract-only	Empirical basis for predicting communication overhead in distributed LLM training, informing when interconnect optimizations (RDMA, collectives) matter most.
samsi2020benchmar	benchmark	Data-parallel deep learning training on shared HPC systems comparing Ethernet and OmniPath fabrics with GPUDirect and NCCL; also CFD workloads	Impact of interconnect hardware and network software primitives on training time	Using Ethernet-based networking in shared HPC systems does not have a significant effect on training times for commonly used DNN architectures or traditional HPC applications such as CFD compared with OmniPath (specific numbers not reported in abstract).	Shared-system noise; older hardware generation; data-parallel workloads only.	moderate	abstract-only	Benchmark evidence that for modest data-parallel workloads interconnect choice (Ethernet vs OmniPath) matters little, contrasting with today's bandwidth-hungry LLM training.
chen2026failures	theoretical	Information-theoretic analysis of AllReduce under asymmetric bandwidth; OptCC validated in SimAI simulations of practical failures with up to 50% bandwidth loss; H100/B100-class multi-NIC servers	AllReduce completion time overhead relative to fault-free optimum	Derives the first information-theoretic lower bound on AllReduce time under asymmetric bandwidth (unavoidable overhead only O(1/p) when the straggler retains at least half its bandwidth) and OptCC completes AllReduce within 2-6% of NCCL's fault-free ring performance, versus up to 57% overhead for the state of the art.	Validation is simulation-based (SimAI); bound requires the straggler to retain at least half bandwidth; primarily single-straggler analysis.	high	full-text	Quantifies and nearly eliminates the AllReduce penalty of NIC-failure rerouting (e.g., PXN-style failover) that otherwise throttles entire data-parallel training collectives.
wang2025reliable	systems paper	Two 8-GPU H100 InfiniBand servers plus large-scale ML simulators modeling hundreds of GPUs with diverse NIC failure patterns	Training/inference overhead under failures and failover robustness versus baselines	R2CCL provides lossless, low-overhead multi-NIC failover with less than 1% training and less than 3% inference overhead, outperforming AdapCC by 12.18x and DejaVu by 47x, in a context where network faults can waste 10-15% of GPU hours.	Physical testbed is small (two servers); large-scale results come from simulation; requires multi-NIC hardware.	high	abstract-only	Fault-tolerant collective library exploiting multi-NIC RDMA connection migration to prevent job-killing timeouts and checkpoint rollbacks in InfiniBand-based training clusters.
pan2025comprehe	survey	GPU collective deadlock phenomena, causes, detection, and prevention in distributed deep learning	Qualitative taxonomy of deadlock scenarios and available solutions	Surveys GPU collective deadlocks that arise when multiple collectives circularly wait for each other and concludes only ad-hoc consistent-ordering methods exist for specific scenarios, with no general effective solution (specific numbers not reported).	Survey-level treatment; no quantitative measurements; fast-moving field.	moderate	abstract-only	Catalogues correctness hazards (deadlocks) in collective communication stacks that practitioners must handle when orchestrating interconnect-heavy LLM training.
arzani2023rethinki	theoretical	TE-CCL evaluated on GPU topologies including Azure NDv2 (2-chassis) and larger topologies; compared with TACCL, SCCL, and heuristics; BERT/DeepLight idle-time context	Schedule quality (finish time / bytes sent) and solver scalability	TE-CCL formulates collective scheduling as a scalable multi-commodity flow MILP, improving TACCL's solution quality by a minimum of 2x in many scenarios while scaling to larger topologies where SMT-based approaches fail, in a context where GPUs idle 11% (BERT) and 63% (DeepLight) without good schedulers.	MILP/LP relaxations may sacrifice optimality on some inputs; evaluation on specific topologies; headline is schedule quality, not measured runtime.	high	full-text	Recasts ML collective scheduling as a traffic-engineering-style flow problem, producing scalable, higher-quality schedules for AllToAll/AllGather/AllReduce on GPU fabrics.
hamidouch2025initiate	systems paper	NCCL 2.28 Device API GPU-Initiated Networking (GIN) architecture; DOCA GPUNetIO kernel-initiated (GDAKI) and Proxy backends; DeepEP integration on multi-node GPU clusters	Qualitative: operation latency and CPU offload; point-to-point microbenchmarks and DeepEP application-level benchmarks	GIN enables device-initiated RDMA within NCCL's unified runtime via kernel-initiated and proxy backends, eliminating CPU orchestration for MoE dispatch/combine and demonstrating practical integration with DeepEP (specific numeric results not reported in read portions).	Requires NCCL 2.28+ and ConnectX-6 Dx or newer NICs for the GDAKI path; proxy backend trades latency for compatibility; specific numbers not captured in read portions.	moderate	full-text	Documents the hardware/software path for GPU-initiated RDMA (GPUDirect Async / DOCA GPUNetIO), central to low-latency MoE all-to-all on InfiniBand/RoCE fabrics.
hwang2022tutel	systems paper	Flex (Tutel) MoE stack on Azure A100 clusters from 16 to 2,048 GPUs; SwinV2-MoE vision model compared with Fairseq	MoE layer speedup and end-to-end training/inference speedup	Flex delivers 4.96x (16 GPUs) and 5.75x (2,048 GPUs) single-MoE-layer speedups over the prior state of the art (up to 3.11x at 128 GPUs) and 1.55x/2.11x end-to-end training/inference speedups over Fairseq, using flexible 2D hierarchical all-to-all and zero-cost adaptive parallelism/pipelining.	Vision-MoE focus; speedups depend on token-routing imbalance; requires identical parameter/data layout design.	high	full-text	Adaptive parallelism and hierarchical all-to-all for MoE, addressing dynamic token-routing traffic patterns on A100 clusters relevant to LLM-scale expert parallelism.
rajbhanda2022deepspee	systems paper	DeepSpeed-MoE training and inference on 128 NVIDIA A100 GPUs (Azure ND A100); 1.3B+MoE-128 vs 6.7B dense; 300B training tokens	Training throughput, inference latency/cost, model size reduction	DeepSpeed-MoE achieves a 5x training compute reduction (372 vs 70 training samples/sec on 128 A100s), compresses MoE model size by up to 3.7x, and delivers 7.3x better inference latency and cost than existing MoE solutions and up to 4.5x faster, 9x cheaper inference than quality-equivalent dense models.	Gains are architecture-level rather than interconnect-specific; inference noted as memory-bandwidth bound; dense-baseline comparisons are model-specific.	high	full-text	Establishes MoE all-to-all communication as a first-class cost in large-model training/inference systems, motivating interconnect- and RDMA-aware expert-parallel design.
deepseeka2024deepseek	systems paper	DeepSeek-V2 MoE LLM (236B total / 21B active parameters), 128K context, pretrained on 8.1T tokens	Training cost, KV cache size, generation throughput	Compared with DeepSeek 67B, DeepSeek-V2 saves 42.5% of training costs, reduces KV cache by 93.3%, and boosts maximum generation throughput to 5.76x, using Multi-head Latent Attention and DeepSeekMoE with device-limited routing to bound MoE-related communication costs.	Model technical report rather than interconnect study; communication control is architectural (routing); cluster hardware details limited.	high	full-text	Production MoE architecture whose device-limited routing and communication-balance losses directly shape all-to-all traffic on GPU interconnects during training and inference.
dai2024deepseek	systems paper	Not reported - the local abstract record contains citation metadata only (Dai et al., ACL 2024, DeepSeekMoE paper)	Not reported	Not reported - the local abstract file contains only bibliographic metadata, so no quantitative findings are available locally.	Abstract record is citation-only; no text or numbers available in the local corpus.	low	abstract-only	Citation-only record of the DeepSeekMoE architecture paper; relevant as MoE routing/expert-specialization context for all-to-all communication in LLM training.
deepseeka2024deepseek2	systems paper	DeepSeek-V3 MoE LLM (671B total / 37B active parameters) pretrained on 14.8T tokens on a 2048-H800-GPU cluster with InfiniBand and NVLink	Training cost (H800 GPU hours), all-to-all communication overhead, pipeline efficiency	DeepSeek-V3's full training costs only 2.788M H800 GPU hours (180K H800 GPU hours per trillion tokens, i.e., 3.7 days per trillion on 2048 H800s), using DualPipe computation-communication overlap and InfiniBand/NVLink-optimized cross-node all-to-all kernels to achieve near-zero all-to-all communication overhead.	Single-cluster, single-vendor hardware report; numbers specific to H800 plus the proprietary IB/NVLink topology; generalizability to other fabrics unproven.	high	full-text	Flagship evidence that computation-communication overlap plus IB/NVLink-optimized all-to-all kernels make 671B-scale MoE training communication-near-free and cost-effective.
nie2024locality	systems paper	RoBERTa-MoE, GPT-MoE, and Swin-MoE training on four A100 servers with 200 Gb/s cross-machine bandwidth	All-to-all communication ratio of training time; training speedup	All-to-all communication averages 45% (up to 67%) of MoE training time, and LSH-MoE's locality-sensitive-hashing token compression with residual error compensation achieves 1.28x-2.2x training speedups across language and vision MoE models.	Lossy compression requires error compensation; evaluated at four-node scale; gains depend on token similarity in the workload.	high	full-text	Quantifies all-to-all as the dominant cost in MoE training (45%+ of time on 200 Gb/s fabrics) and shows token-similarity compression as a communication-volume mitigation.
zhang2026hiding	systems paper	MoE training (OLMoE, DeepSeek-MoE-16B) under expert parallelism with all-to-all communication	MoE layer execution time / training efficiency; all-to-all communication hiding	HALO reduces execution time on OLMoE by 26.1%, 22.9%, and 22.0% on average versus PipeMoE, MPMoE, and ScMoE respectively, and by 15.2%, 13.1%, and 20.4% on DeepSeek-MoE-16B.	Shadow-expert replication adds memory overhead; benefit hinges on routed-token locality and a sufficiently large local computation window.	high	abstract-only	Shows all-to-all communication in expert-parallel MoE training can be hidden via locality-aware overlap and shadow experts, a key technique for interconnect-bound LLM training.
qi2026feplb	systems paper	GLM-5 MoE layers (128 experts, no auxiliary loss, up to 16 H100 GPUs, EP up to 8)	Token/GEMM straggler, EP communication overhead, GPU time waste	FEPLB reduces the token straggler by 51-70% and the GEMM straggler by 50-68% with no measurable EP communication overhead on GLM-5's MoE layers, and at EP=8 achieves 2x lower token straggler than FasterMoE, where load imbalance alone wastes on average 18.6% of GPU time per MoE layer.	Requires NVIDIA Hopper NVLink Copy Engine and only rebalances within the intra-node NVLink domain; evaluated on a single model family.	high	full-text	Exploits the NVLink Copy Engine as a near-free intra-node communication channel parallel to compute kernels, informing NVLink usage for MoE load balancing in LLM training.
qin2026reducing	systems paper	MoE model training with hybrid parallelism across Pods in multi-tenant NPU clusters (models/topologies not specified in abstract)	Cross-Pod traffic volume, communication time	A two-stage Best-Fit + neighborhood-search placement scheme for MoE hybrid-parallel training reduces cross-Pod traffic by 35.9% and cuts communication time by 18.7% compared to state-of-the-art methods.	Abstract-only record; NPU- and multi-tenant-specific; relies on quantifying cross-Pod communication overhead as a 0-1 integer quadratic problem.	moderate	abstract-only	Addresses cross-Pod placement to mitigate intra-/inter-Pod bandwidth and latency discrepancies for MoE training, relevant to interconnect-aware job placement.
zhang2026dynamic	systems paper	MoE LLMs (e.g., DeepSeek-v3) on multi-GPU NVLink/NVSwitch systems; simulated GH200 NVL32	End-to-end and MoE-layer speedup, redundant traffic reduction	DySHARP achieves up to 1.79x speedup over the state of the art via dynamic in-switch computing (NVLink SHARP extension), where MoE communication consumes 50-80% of execution time and profiling shows near 50% communication redundancy.	Requires ISA, microarchitecture, and switch changes (hardware co-design); evaluation uses simulated GH200 NVL32 hardware.	high	full-text	Extends NVLink SHARP in-switch reduction to irregular dynamic MoE traffic, showing in-switch computing can cut redundant dispatch/combine transfers in LLM training.
zhu2025megascal	systems paper	Not reported (partial abstract; MoE LLM inference context)	GPU utilization, operational cost (motivational only)	Not reported - the partial abstract only motivates that MoE's sparsely activated architecture shifts FFNs from compute-intensive to memory-intensive during inference, lowering GPU utilization and raising cost.	Abstract is truncated/partial with no methods or results; full text unavailable.	low	abstract-only	Partial abstract frames memory-bound MoE inference efficiency, relevant to memory/interconnect pressure in LLM serving.
cai2024shortcut	systems paper	SwinV2-MoE-S vision and language MoE models; 8x A30 and 8x A800 GPU nodes	Training/inference speedup, All-to-All communication share, overlap ratio	ScMoE achieves 1.49x training and 1.82x inference speedups over a top-2 MoE baseline with up to 100% communication-computation overlap, where All-to-All takes 60% of block time on 8x A30 but drops to 15% on 8x A800 due to ~6x higher NVLink bandwidth.	Changes the MoE architecture (shortcut connections), which may affect gating/model quality; inter-node Ethernet still limits multi-node scaling.	high	full-text	Quantifies how NVLink bandwidth (A30 vs A800) changes All-to-All overhead and shows shortcut-connected MoE decouples communication from computation for overlap.
wang2022topoopt	systems paper	12-node direct-connect prototype with RDMA forwarding at 100 Gbps; large-scale simulations of real distributed training models	DNN training time, AllReduce and model-parallel transfer efficiency	TopoOpt reduces DNN training time by up to 3.4x versus similar-cost Fat-Tree interconnects in large-scale simulations, validated on a fully functional 12-node direct-connect prototype with RDMA forwarding at 100 Gbps.	Requires direct-connect (optical circuit switch) fabric and host-based RDMA forwarding via NIC NPAR; large-scale results are simulation-based.	high	full-text	Pioneers co-optimizing network topology with AllReduce traffic mutability for DNN training, foundational for topology-aware interconnect design (RDMA/OCS).
namkoong2025sortingh	systems paper	Various LLMs on diverse multi-GPU environments (topology-aware)	Inference latency, speedup over single-GPU baseline	SortingHat, using dominator analysis plus MILP-based scheduling that exploits both model parallelism and data-transfer parallelism, achieves an average speedup of 2.28x and up to 2.96x over the single-GPU TVM baseline.	MILP-based search may not scale to very large models; inference-only focus; baselines limited to TVM.	moderate	abstract-only	Shows multi-GPU LLM inference gains depend on co-scheduling data-transfer parallelism with system topology, relevant to interconnect-aware scheduling.
chang2026spiderfl	systems paper	Not reported (citation-only record)	Not reported	Not reported - only bibliographic metadata (ACL 2026) is available; the abstract file contains no content.	Record contains citation metadata only; no abstract or full text to extract evidence from; design inferred from title.	low	abstract-only	Citation-only record (SpiderFL, ACL 2026); likely federated-learning communication/scheduling work but no extractable interconnect evidence.
ye2023workload	survey	Published DL scheduling research for training and inference workloads in GPU datacenters	Scheduling objectives, resource utilization, throughput (qualitative)	Survey of DL workload scheduling in GPU datacenters covering training and inference schedulers, objectives, and resource-utilization mechanisms; no single quantitative result (survey).	Survey-level; no new measurements; specific numbers not summarized in the abstract.	high	abstract-only	Provides a taxonomy of GPU-datacenter scheduling including communication- and resource-aware methods, context for interconnect-aware training scheduling.
yu2022scheduli	theoretical	Analytical model plus experiments on GPU clusters with multiple ring-all-reduce training jobs (details in full text)	Makespan of co-located ring-all-reduce training jobs	Not reported in abstract - proposes an analytical model of RAR communication overhead and contention and an SJF-BCO (Smallest Job First with Balanced Contention and Overhead) approximation algorithm to minimize makespan.	Abstract truncated; theoretical results may rely on simplifying contention assumptions.	moderate	abstract-only	Models communication contention between co-located all-reduce training jobs, relevant to RDMA/collective scheduling in GPU clusters.
dong2026phase	systems paper	LLM training communication phases on OCS-reconfigurable optical networks (scale not stated)	Communication time over static vs reconfigurable optical networks	A phase-aware optical network using OCSs to reconfigure into phase-optimal topologies with Rabenseifner/Bruck mappings and slot amortisation achieves 37.5% faster communication than static optical networks under established LLM training configurations.	Short OFC abstract; evaluation methodology (simulation/testbed) and scale not stated.	moderate	abstract-only	Evidence that reconfigurable optical (OCS) topologies can accelerate LLM training collectives by matching topology to the communication phase.
cho2023logical	systems paper	8-GPU NVIDIA DGX-1 proof-of-concept	AllReduce communication time, end-to-end training time	C-Cube chains and overlaps tree-AllReduce phases with forward computation via gradient queuing and adds physical-topology detour/double-tree variants, improving communication and overall performance versus non-overlapped tree algorithms (magnitude not reported in abstract).	Abstract gives no numeric speedup; prototype limited to 8 GPUs.	moderate	abstract-only	Demonstrates logical/physical topology-aware AllReduce with computation chaining on an NVLink-based DGX-1, relevant to collective optimization.
won2023astra	simulation	Simulated case studies of emerging distributed training models/platforms (wafer-scale nodes, multi-dimensional topologies, disaggregated memory)	Training throughput/time via analytical performance estimation	Not reported - ASTRA-sim2.0 is a simulation infrastructure extension supporting arbitrary model parallelization, parameterizable multi-dimensional heterogeneous topologies, and in-network collective/memory modeling for design-space exploration at scale.	Simulation-only; accuracy depends on analytical performance models.	high	abstract-only	Provides an open simulator for interconnect/topology-aware distributed training design-space exploration (NVLink, InfiniBand, collectives, disaggregated memory).
jiang2024megascal	field study	175B-parameter LLM training on 12,288 GPUs (production system at >10,000-GPU scale)	Model FLOPs Utilization (MFU), training stability, fault tolerance	MegaScale achieves 55.2% MFU when training a 175B LLM on 12,288 GPUs, improving MFU by 1.34x compared to Megatron-LM.	Single proprietary stack; MFU depends on specific model/hardware; stability fixes are largely empirical.	high	full-text	Production evidence that communication-computation overlap, network tuning, and observability sustain 55% MFU at 12k+ GPUs - a reference point for interconnect-bound LLM training.
meng2025astral	field study	Production LLM training/inference datacenter infrastructure deployed over 18 months for multiple customers	Interconnect architecture (same-rail tier-2), failure diagnosis, operator timeline forecasting	Not reported in abstract - Astral deploys a same-rail interconnection network on tier-2 for LLM training with full-stack monitoring and operator-granular Seer forecasting for fault diagnosis and network upgrades.	No quantitative results in the abstract; deployment-specific insights.	moderate	abstract-only	Field experience with same-rail network design for LLM training clusters, relevant to datacenter interconnect architecture.
jouppi2023opticall	field study	TPU v4 supercomputer, 4096 chips, production since 2020	Achieved FLOPS fraction, relative performance/power, OCS cost and power share	TPU v4 with OCS-reconfigurable twisted 3D torus trains a large language model at an average of ~60% of peak FLOPS, outperforms TPU v3 by 2.1x (2.7x per Watt) and A100 by 1.2-1.7x, while OCSes cost <5% of system cost and <3% of system power.	Google-specific deployment; OCS reconfiguration overheads and topology choices not fully quantified here.	high	abstract-only	Seminal evidence that OCS-based reconfigurable optical interconnects are cheaper and lower power than InfiniBand and sustain ~60% peak FLOPS for LLM training.
liu2023lightwav	field study	4096-chip TPU v4 ML superpod (>1 ExaFLOP) with reconfigurable lightwave fabric	System availability, model training performance, cost share	Lightwave fabric (OCS + WDM/circulators) deployment provides up to 3x better system availability and model-dependent performance improvements of up to 3.3x versus a static fabric, at less than 6% of total system cost.	Google-internal deployment; performance gains are model-dependent.	high	abstract-only	Shows reconfigurable optical lightwave fabrics improve availability and LLM training performance, supporting OCS-based interconnect arguments.
shou2025infinite	systems paper	High-bandwidth domain (HBD) architectures for LLM training (NVL-72, TPUv3, Dojo, TPUv4)	Scalability, cost, fault resiliency of HBD interconnects (qualitative)	Not reported - partial abstract only surveys HBD architectures (switch-centric NVL-72, GPU-centric TPUv3/Dojo, hybrid TPUv4) and their scalability, cost, and fault-propagation tradeoffs.	Truncated abstract; no results available.	low	abstract-only	Frames the high-bandwidth-domain (NVLink-domain) scalability/cost/fault tradeoff space for LLM training interconnects.
feng2025mercury	systems paper	FPGA-based real-time prototype of an optical-switched network for ML training	Reconfiguration granularity, training acceleration	Mercury, combining collaborative optical timeslot and circuit switching, demonstrates nanosecond-granularity reconfiguration and model-training acceleration on a real-time FPGA prototype (speedup magnitude not reported in abstract).	Short OFC abstract; acceleration magnitude and scale not reported.	moderate	abstract-only	Evidence for nanosecond-scale reconfigurable optical switching as an interconnect option for ML training.
yang2025multi	systems paper	Multi-GPU systems; Rodinia benchmarks and LLM training kernels	Speedup, energy consumption, chip area	BEAM, a multi-channel in-package optical interconnect architecture for multi-GPU systems, achieves a speedup of 1.14-1.9x and reduces energy by 29-44% compared to electrical-interconnected systems and state-of-the-art schemes, with comparable chip area.	Architecture-level evaluation; optical packaging/thermal realities not covered in the abstract.	moderate	abstract-only	Argues for in-package optical (vs electrical) GPU interconnects for bandwidth and power at scale, relevant to NVLink-alternative roadmaps.
zhou2026packaged	survey	Articles in the APNexus co-packaged optics (CPO) theme issue	Not applicable (editorial overview)	Not reported - editorial overview by APNexus editors presenting the articles in the CPO theme issue.	Editorial content only; no data or measurements.	low	abstract-only	Points to co-packaged optics as a packaging-level interconnect direction for AI/network systems.
nabavinej2020overview	survey	Published DNN accelerator interconnection designs (ASIC and non-ASIC, NoC, reconfigurable, in/near-memory)	On-chip bandwidth, power, computing flexibility (qualitative)	Not reported - comprehensive survey of on-chip interconnection methods for DNN accelerators covering different interconnection methods, reconfigurable DNN computing, and emerging in/near-memory processing technologies.	Survey-level; no new quantitative measurements.	high	abstract-only	Background survey on on-chip NoC/interconnect design tradeoffs for DNN accelerators, upstream of scale-out interconnects like NVLink/RoCE.
jung2025compute	theoretical	Large-scale AI hardware and data center designs (GB200/Blackwell, hierarchical clusters); parametric evaluations	Communication overhead share of training time, memory/interconnect bandwidth (TB/s, GB/s, Gb/s)	Technical report citing that inter-GPU communication accounts for 35-70% of total training time in large-scale AI deployments, describing GB200 with 192 GB HBM3e at up to 8 TB/s per GPU and ~900 GB/s NVLink C2C to 480 GB LPDDR5X, and proposing CXL-over-XLink (UALink/NVLink) disaggregation to reduce long-distance transfers.	Position/report piece; many figures cited from industry analyses; proposed architecture not fully validated in production.	moderate	full-text	Useful synthesis of the communication-tax (35-70% of training time) and the NVLink/CXL/UALink bandwidth hierarchy for LLM infrastructure.
gouk2023memory	systems paper	Host processor complex + remote memory over CXL.mem vs RDMA (diverse real-world workloads)	Memory access performance (relative to RDMA-based pooling)	DirectCXL, which directly connects a host processor to remote memory over CXL.mem, exhibits around 7x better performance than RDMA-based memory pooling across diverse real-world workloads.	Prototype-based empirical study; CXL hardware ecosystem still nascent at publication.	moderate	abstract-only	Quantifies CXL.mem vs RDMA for memory disaggregation, relevant to KV-cache/memory pooling for LLM serving.
gu2025enabled	systems paper	LLM inference workloads (autoregressive decoding, KV cache, context windows up to 1M tokens)	Memory bandwidth demand vs compute throughput (qualitative)	Not reported - abstract motivates that LLM inference is memory-bandwidth-bound due to low operational intensity and large per-prompt KV caches, and that compute-throughput-optimized GPUs/TPUs are poorly utilized for such tasks.	Partial/motivational abstract; no methods or results.	low	abstract-only	Frames memory-bandwidth-bound LLM inference as the driver for high-bandwidth memory and interconnect systems.
ko2026internal	systems paper	CXL memory expander workloads (cold/hot page mixes)	Effective memory capacity, access speedup	IBEX, a block-level compression scheme for CXL memory expanders that compresses only cold data with internal bandwidth-efficient block management, achieves average 1.28x-1.40x speedups over state-of-the-art promotion-based block-level approaches.	Compression adds overhead for cold-data access; compared against compression baselines rather than full systems.	moderate	abstract-only	Addresses bandwidth/capacity limits of CXL memory expanders, an emerging tier for LLM KV-cache offload.
quinn2025compute	systems paper	CXL memory expanders with near-memory accelerators for dense retrieval (RAG)	Dense retrieval acceleration, memory access interference (qualitative)	Not reported in abstract - IKS, a compute-enabled CXL memory expander with lightweight near-memory accelerators leveraging high internal bandwidth, accelerates dense retrieval for RAG while supporting spatial and temporal multi-tenancy.	No quantitative results in the abstract.	moderate	abstract-only	Combines CXL memory expansion with near-memory compute for RAG retrieval, relevant to memory-centric LLM inference.
fang2025dynamic	theoretical	Heterogeneous memory (HBM + LPDDR5X/NVLink) LLM inference systems; simulation-based validation	Aggregated bandwidth utilization, inference throughput upper bound	A formal treatment of dynamic KV-cache placement across HBM and off-package DRAM (NVLink/LPDDR5X) derives a theoretical upper bound achieving up to 5.87x higher throughput than static placement in simulation, with NVLink 4.0 at up to 100 GB/s per link and GH200 NVLink-C2C at ~900 GB/s.	Theoretical bound, not a deployed policy; assumes knowledge of future access patterns.	moderate	full-text	Quantifies headroom in KV-cache placement across HBM/off-package DRAM tiers, informing memory-interconnect design for LLM inference.
wang2022enabling	systems paper	Distributed DL training with disaggregated memory (cache-coherent interconnect + MPI-like collectives)	Training time vs state-of-the-art MPI AllReduce	COARSE, a disaggregated memory extension built on cache-coherent interconnect protocols with decentralized parameter synchronization and dynamic tensor routing, achieves up to 48.3% faster DL training than state-of-the-art MPI AllReduce communication.	Requires CCI-capable hardware; evaluation scope (cluster size, models) not stated in abstract.	moderate	abstract-only	Shows cache-coherent interconnects plus disaggregated memory can beat MPI AllReduce for parameter synchronization, relevant to RDMA/CCI tradeoffs.
li2023memory	systems paper	158 cloud production workloads; CXL memory pools spanning 8-16 sockets	DRAM cost reduction, performance relative to same-NUMA-node allocation	Pond, a CXL-based memory pooling system using small pools (8-16 sockets) and ML-based allocation prediction, reduces DRAM costs by 7% while keeping performance within 1-5% of same-NUMA-node VM allocations across 158 workloads.	Cloud-specific; modest 7% cost saving; requires accurate ML prediction models.	high	abstract-only	Establishes that small CXL pools meet cloud performance goals, informing memory-pooling economics for LLM serving.
ganjihal2026topology	theoretical	Disaggregated prefill/decode GPU pools; 70B-class models; analytical bandwidth models	KV cache transfer latency, aggregate bandwidth demand	Projected analysis: KV-cache transfer for a 70B model is 2.6 GB per request (exceeding 100 GB/s aggregate at production scale), bandwidth between GPUs varies 72x (900 GB/s NVLink, 50 GB/s InfiniBand, 12.5 GB/s TCP), and topology-aware transport with pipelined transfer hides 60-85% of latency, giving 3-18x transfer-latency reduction over uniform RDMA.	Author-flagged: no experimental validation possible (multi-node heterogeneous interconnects and CXL 3.0 hardware unavailable); analytical projections only.	moderate	full-text	Quantifies the NVLink/InfiniBand/TCP bandwidth hierarchy (72x spread) that disaggregated inference systems must exploit, critiquing uniform-RDMA designs like DistServe/Splitwise/Mooncake.
zhong2024distserv	systems paper	Popular LLMs and applications on GPU serving clusters (A100-class hardware)	Served request rate within TTFT/TPOT SLO constraints	DistServe, which disaggregates prefill and decoding and places phases according to cluster bandwidth, serves 7.4x more requests or meets 12.6x tighter SLOs than state-of-the-art systems while staying within latency constraints for >90% of requests.	Requires bandwidth-aware placement; gains depend on workload and latency requirements.	high	full-text	Canonical prefill/decode disaggregation paper showing placement must account for interconnect bandwidth to minimize disaggregation communication.
patel2023splitwis	systems paper	A100/H100 GPU clusters; LLM inference characterization and cluster design (simulator-based)	Throughput, cost, power, KV state-transfer time	Splitwise phase-splitting clusters achieve 1.4x higher throughput at 20% lower cost, or alternatively 2.35x more throughput under the same cost and power budgets, with KV state transferred over fast back-plane interconnects (e.g., NVLink 50/100 Gbps, InfiniBand 200/400 GBps on A100/H100).	Cluster designs rely on a simulator; state-transfer overhead grows with model size and disaggregation distance.	high	full-text	Quantifies throughput/cost gains of phase splitting and the role of fast interconnects (NVLink/InfiniBand) in KV state transfer between phases.
qin2025mooncake	field study	Kimi LLM serving platform, thousands of nodes, NVIDIA A800/H800 clusters, real traces	Effective request capacity under SLOs, tokens processed per day	Mooncake, a KVCache-centric disaggregated serving platform processing over 100 billion tokens daily across thousands of nodes, increases effective request capacity by 59-498% versus baseline methods under SLOs and handles 115% (A800) and 107% (H800) more requests than previous systems.	Production-specific; benefits concentrated in long-context scenarios.	high	abstract-only	Production proof that disaggregated KVCache using spare CPU/DRAM/SSD/NIC resources scales LLM serving, with data-movement implications across interconnects.
wu2024loongser	systems paper	LoongServe LLM serving system evaluated on GPU clusters with NVLink and InfiniBand interconnects; real-world workload datasets (ShareGPT et al.); 1M-token context requests on 7B-class models	max throughput, KV cache migration overhead, per-token latency	LoongServe improves maximum throughput by up to 3.85x over chunked prefill and 5.81x over prefill-decoding disaggregation, with a single 1M-input request needing 488GB of KV cache.	Elastic scaling incurs KV cache migration cost over high-bandwidth interconnects; scheduling must fit tens-of-ms iteration budgets; evaluations on specific GPU clusters may not generalize.	high	full-text	Shows elastic sequence parallelism reduces KV-transfer/communication overhead in long-context serving, quantifying the interconnect cost of KV migration across NVLink/InfiniBand.
liu2025serverle	systems paper	ServerlessPD serverless LLM inference system with RDMA-based remote fork across machines	cold-start latency, KV cache state transfer cost, autoscaling latency	RDMA-co-designed remote fork enables near-instant autoscaling and zero-copy state (KV cache) transfer for serverless prefill-decode disaggregation (specific speedups not reported in abstract).	Abstract-only; no quantitative results or hardware scale reported in the available record.	moderate	abstract-only	Demonstrates RDMA kernel-integrated remote fork as a mechanism for zero-copy KV state migration in disaggregated serverless LLM serving.
liu2025lmcache	systems paper	LMCache KV caching layer with vLLM and SGLang; enterprise deployments; Qwen3-32B on 2x H100; 500GB CPU DRAM tier	throughput, TTFT, KV cache transfer/offload bandwidth, prefix cache hit ratio	Combining LMCache with vLLM achieves up to 15x throughput improvement, and real-world usage shows remote KV fetching benefits prefill delay while context truncation halves prefix cache hit ratio.	Evaluation uses stretched real traces run within one hour; results depend on storage/network backends (NFS, WEKA, GPU-Direct Storage, Mooncake Store, NIXL, S3, InfiniStore, Valkey).	high	full-text	Provides an open-source KV cache layer spanning GPU/CPU/storage/network tiers, quantifying gains of cross-engine KV movement for PD disaggregation and prefix reuse.
yang2026beyond	systems paper	DOPS framework on heterogeneous NPU-PIM systems; LLM inference workloads	geometric-mean end-to-end latency speedup, weight layout overhead	Bifocal scheduler achieves 1.20x-2.23x geometric-mean speedups over prefill-decode (PD) disaggregation baselines and WLA adds a further 1.28x-1.33x via hardware-efficient weight layouts.	Targets edge-oriented NPU-PIM platforms rather than GPU clusters; results are largely from performance-model-driven scheduling with limited real-system scale.	high	full-text	Argues PD disaggregation alone is insufficient and models communication/placement costs (all-reduce, gather, scatter) for operator scheduling across heterogeneous interconnects.
wei2024parallel	systems paper	DeTransformer distributed in-situ Transformer inference on 4 edge devices with weak interconnect	distributed inference latency, communication volume, accuracy	DeTransformer reduces distributed inference latency by up to 2.81x versus the SOTA approach on 4 devices while maintaining accuracy and model size.	Abstract-only; edge scenario with weak interconnect, small scale (4 devices), and no absolute bandwidth numbers reported.	moderate	abstract-only	Shows block-parallel inference restructuring to cut communication frequency under weak interconnects, relevant to interconnect-constrained distributed inference.
li2021terapipe	systems paper	TeraPipe token-level pipeline parallelism; GPT-3 models (1B-175B) on AWS cluster of 48 p3.16xlarge instances	per-iteration training latency, TFLOPs/GPU, speedup vs state-of-the-art model parallelism	TeraPipe speeds up GPT-3 175B training by 5.0x on 48 p3.16xlarge instances (1.40x for GPT3-13B, 1.88x/1.56x for GPT3-44B configurations).	Speedup vanishes when optimal slicing only uses the batch dimension; requires Transformer autoregressive structure; depends on cluster network bandwidth.	high	full-text	Exploits token-dimension pipelining to cut communication stalls in model-parallel LLM training; contrasts with allreduce-based Megatron-LM synchronization.
li2023sequence	systems paper	not reported (abstract record contains only citation metadata)	not reported	Proposes sequence parallelism for long-sequence training from a system perspective; no quantitative results available in the record.	Only citation metadata available; no abstract text, numbers, or methodology accessible locally.	low	abstract-only	Positional reference for sequence parallelism (SP) as an interconnect-relevant alternative to tensor parallelism for long-sequence LLM training.
brandon2023striped	systems paper	Striped Attention vs Ring Attention; A100 GPUs and TPUv4s; 1B/3B/7B causal transformers at 256k-786k sequence lengths	end-to-end training throughput, speedup over Ring Attention	Striped Attention achieves up to 1.45x end-to-end throughput over Ring Attention at 256k sequence length on A100/TPUv4 and 1.65x speedups on 16 TPUv4 chips at 786k.	Improvements specific to causal (triangular-mask) attention; more carefully optimized Ring/Striped implementations left to future work.	high	full-text	Improves load balance of ring-based context-parallel attention (KV communication overlapped with compute), relevant to long-context training communication patterns.
gu2024loongtra	systems paper	LoongTrain 2D-Attention; cluster of 8 GPU servers x 8 NVIDIA Ampere GPUs, NVLink intra-node, 4x 200Gb/s HDR InfiniBand NICs inter-node; sequence lengths to 1M	Model FLOPs Utilization (MFU), end-to-end training speed, SeqAlltoAll and P2P communication time	LoongTrain improves MFU by up to 2.88x over DeepSpeed-Ulysses and Megatron Context Parallelism, with Megatron-CP hitting out-of-memory at 1M sequence length.	Double-Ring-Attention gains depend on NIC count and device placement; GQA KV-replication trade-offs limit some configurations.	high	full-text	Combines head-parallel and context-parallel attention to use all inter-node NICs (HDR InfiniBand) efficiently, quantifying SeqAlltoAll vs P2P communication trade-offs.
ghadia2026untied	systems paper	UPipe (Untied Ulysses) headwise-chunked context parallelism; Llama3-8B on 8x H100 and Qwen3-32B on 16x H100; up to 8M-token contexts	intermediate tensor (QKV/all-to-all) activation memory, tokens/second/GPU throughput, max context length	UPipe cuts intermediate attention-layer tensor memory by up to 87.5% for 32B transformers and supports 5M-token context on a single 8xH100 node (25% beyond FPDT), scaling to 8M on 16 H100s at throughput comparable to baselines.	More head chunks increase runtime (memory-throughput trade-off); authors note limitations in the appendix scope section; multi-node configs use Ulysses+ring hybrids.	high	full-text	Shows headwise chunking to shrink all-to-all activation buffers in context-parallel training, addressing the memory side of long-context collective communication.
jiang2025addressi	systems paper	DCP dynamic context-parallel training framework; blockwise data/computation partitioning; micro-benchmarks plus end-to-end training	attention acceleration, end-to-end training speedup, communication overhead	DCP accelerates attention by 1.19x-2.45x under causal masks and 2.15x-3.77x under sparse attention, with 0.94x-1.16x (causal) and 1.00x-1.46x (sparse) end-to-end training speedups.	Abstract-only; gains depend on sequence-length variability and attention-pattern sparsity of the training data.	moderate	abstract-only	Motivates dynamic (rather than static) context-parallel partitioning to cut unnecessary communication for variable-length long-context training.
narayanan2021scale	systems paper	Megatron-LM tensor+pipeline+data parallelism composition; 1-trillion-parameter model on 3072 GPUs	training throughput (petaFLOP/s), per-GPU fraction of theoretical peak, scaling efficiency	Interleaved pipelining improves throughput by 10+% at comparable memory footprint, enabling 1T-parameter training at 502 petaFLOP/s on 3072 GPUs (52% of per-GPU theoretical peak).	Abstract-only; scaling results assume specific cluster interconnect; naive tensor/pipeline composition has scaling issues at thousands of GPUs.	moderate	abstract-only	Canonical reference on composing parallelisms to reach thousands of GPUs, setting the communication-efficiency baseline that interconnect-aware schedules target.
wang2020whale	systems paper	Whale framework on production Alibaba cluster of 512 GPUs (V100/P100 heterogeneous); M6 multimodal model with >10T parameters	training throughput, scaling efficiency, speedup from hardware-aware balancing	Whale trains the >10T-parameter M6 model on 512 GPUs with 95% scaling efficiency from 8 to 32 GPUs and 1.2x-1.4x speedups from hardware-aware load balancing (e.g., NVLink-aware AllReduce placement).	Speedups modest (1.2x-1.4x) for load balancing; evaluation largely on internal Alibaba workloads.	high	full-text	Demonstrates graph-level placement of collectives (e.g., NVLink-based AllReduce within nodes) as a lever for efficient giant-model training on heterogeneous clusters.
sensi2024explorin	benchmark	Three exascale supercomputers - Alps, Leonardo, LUMI - intra-node and inter-node interconnects on up to 4,096 GPUs	intra-node and inter-node interconnect bandwidth, latency, achieved vs peak throughput	Characterization shows untapped interconnect bandwidth and optimization opportunities across network and software layers (absolute bandwidth figures not reported in abstract).	Abstract-only; no numeric results in the record; findings tied to specific hardware generations and software stacks.	moderate	abstract-only	Provides comparative interconnect characterization (multi-Tbps intra-node networks) for GPU supercomputers, informing expectations for large-scale LLM training fabrics.
li2021scale	systems paper	1-bit LAMB compressed communication; BERT-Large pre-training at batch sizes 8K-64K on up to 256 GPUs; Ethernet clusters with V100s	communication volume, end-to-end training time, convergence speed, allreduce share of step time	1-bit LAMB with NCCL backend achieves up to 4.6x communication-volume reduction and up to 2.8x end-to-end speedup with identical convergence, where allreduce can consume up to 91% (and 52% on another cluster) of per-step time.	Compression trades some accuracy risk; extra memory for variance/momentum copies; benefits shrink on high-bandwidth clusters.	high	full-text	Quantifies allreduce-dominated communication on commodity TCP/Ethernet clusters and shows gradient compression as an interconnect-bandwidth mitigation for LLM-scale training.
li2024lossless	systems paper	GComp near-lossless gradient compression for data-parallel DNN training; multi-node GPU clusters	communication volume reduction, training speedup, accuracy preservation	GComp decreases synchronization communication volume by up to 67.1% and improves training speed by up to 1.9x with virtually no accuracy loss.	Abstract-only; effectiveness depends on gradient statistical characteristics and model/workload; compression ratio varies across models.	moderate	abstract-only	Near-lossless gradient compression as a way to relieve interconnect bandwidth pressure in data-parallel LLM-scale training.
li2024driven	systems paper	FreeINA host-driven in-network aggregation; P4-programmable switch + commercial RNICs on a 100Gbps testbed	single-job training speedup, aggregation throughput, switch memory efficiency	FreeINA improves single-job training speedup by 1.20x over ATP and aggregation throughput by 2.65x in multi-job scenarios while keeping RDMA RC compatibility.	Requires programmable switches; evaluation on 100Gbps testbed, not production-scale fabrics; multi-tenant assumptions may not hold in all settings.	moderate	abstract-only	Addresses the collision between in-network aggregation (INA) and RDMA reliable connections for DNN training, relevant to RDMA/RoCE-based training clusters.
segal2022constrai	theoretical	Algorithmic study of constrained in-network computing for distributed workloads in datacenter topologies (tree-like networks)	network congestion (max loaded link), reduction in congestion	An optimal algorithm for tree-like topologies reduces congestion by up to 13x, with even a small fraction of in-network-aggregation-capable devices giving significant gains.	Theoretical bounds for tree-like topologies only; practical switch constraints and multi-tenant dynamics simplified.	moderate	abstract-only	Formal foundation for in-network aggregation (e.g., SHARP-style) deployment, underpinning congestion reduction for distributed ML collectives.
zhu2024meets	survey	Survey of In-Network Computing (INC)-based distributed machine learning systems on programmable switches	qualitative performance improvement analysis, taxonomy of INC-based DML systems	Overview paper: INC accelerates DML training via switch programmability/compute/storage; no quantitative results reported.	Survey without experimental numbers; coverage limited to INC-based DML systems and research directions.	moderate	abstract-only	Survey framing INC (programmable-switch aggregation) as an accelerator for distributed ML, contextualizing switch-offload approaches for collectives.
wang2023blink	systems paper	Blink collective library on DGX-1P/DGX-1V/DGX-2 (NVLink, NVSwitch, PCIe); GPU counts 3-8+; data sizes 1MB-1000MB; cluster traces of 40,000 multi-GPU jobs	model synchronization time, broadcast/AllReduce throughput and latency, end-to-end training time	Blink achieves up to 8x faster model synchronization than NCCL and up to 40% lower end-to-end training time, with ~18GB/s throughput on datasets >10MB and up to 3.32x lower latency / 3.5x higher throughput on DGX-2 NVSwitch AllReduce.	Focused on image-classification-scale training; topology/heterogeneity assumptions (PCIe vs NVLink bandwidth) baked into tree packing; later NCCL versions close some gaps.	high	full-text	Spanning-tree packing over heterogeneous NVLink/PCIe/NIC topologies as an alternative to NCCL ring collectives, directly relevant to collective design for training.
ryohei2022mission	case study	Apollo optical circuit switching (OCS) production deployment in Google datacenters; 3D MEMS-based OCS with circulators and WDM transceivers over four interconnect generations	network bandwidth, scale, cost, OCS switching time, port count	World's first large-scale production OCS deployment uses circulators to double OCS radix and co-designed WDM transceivers to meet datacenter bandwidth/scale/cost targets (specific Gbps numbers not reported in available text).	Available record is the arXiv abstract page only (no full body); switching-time and performance details not extracted.	moderate	abstract-only	Production evidence for optical circuit switching in datacenter networks, a candidate technology for future AI-cluster fabric reconfiguration.
yan2023demystif	benchmark	First true CXL-ready systems: 4th-gen Intel Xeon (SPR) with three CXL memory devices from different vendors; microbenchmarks plus DLRM and other bandwidth-intensive workloads	CXL memory latency and bandwidth vs emulated CXL and DDR, application throughput, page-allocation ratio	Captión, a CXL-memory-aware dynamic page allocation policy, improves memory-bandwidth-intensive application performance by up to 24% versus default NUMA policies, and true CXL devices differ notably from DDR-emulated CXL.	Single-vendor CPU platform; results specific to SPR and the three tested CXL devices; CXL latency still higher than local DDR.	high	full-text	Empirical latency/bandwidth characterization of CXL memory expansion (and CXL vs RDMA remote access), informing memory-disaggregation options for LLM inference.
das2023introduc	survey	Survey of CXL 1.0/2.0/3.0 standards, implementations, and datacenter impact	qualitative: bandwidth scaling with PCIe, latency vs PCIe, ecosystem adoption	CXL provides coherency/memory semantics with bandwidth scaling with PCIe at significantly lower latency than PCIe, adopted by all major CPU/device vendors (no quantitative figures in record).	Available record is the arXiv abstract page only; survey-level without benchmark numbers.	moderate	abstract-only	Reference survey for CXL as an interconnect (memory expansion, pooling, peer-to-peer), a growing alternative/complement to RDMA for disaggregated memory.
angelos2024streamli	case study	ZeroPoint compressed CXL tiered-memory expander aligned with OCP Hyperscale specification; QEMU+FPGA proof of concept	compression ratio, cache-line access latency, TCO reduction	Hardware-accelerated lossless compressed CXL memory tier offers 2-3x compression in nanoseconds and 20-25% TCO reduction, meeting OCP targets of 250ns compressed-block access and <1us tail latency.	Industry position paper with emulation-based PoC; production silicon claims (mid-2024) not independently verified; CXL pool economics contested.	moderate	full-text	Shows compressed CXL tiers to expand effective memory capacity at hyperscale, addressing memory-capacity constraints in AI inference without extra network traffic.
saurabh2024symphony	systems paper	Symphony on 2 nodes x 4 NVIDIA A100 80GB HBM, 256GB DRAM, 4TB SSD per node, 100Gbps Ethernet inter-node; multi-turn chatbot and MetaGPT agent workloads	requests served per second, latency, load imbalance, redundant-token fraction	Symphony handles over 8x the requests of state-of-the-art baselines at similar latency, leveraging advisory requests issued on average 5.8s early (MetaGPT) and avoiding >99% redundant recomputed tokens.	Relies on advisory requests/hints from multi-turn workloads; hierarchical KV migration to host/disk adds complexity; benefits shrink without advisory info.	high	full-text	KV-cache migration off the critical path as an alternative to stateful offloading, illustrating state-transfer overheads across cluster interconnects (100Gbps Ethernet).
wanqian2026weight	systems paper	DWDP on NVIDIA GB200 NVL72 with DeepSeek-R1 (NVFP4 MoE); disaggregated serving, 8K input / 1K output sequences	output tokens/sec/GPU, TTFT, TPS/user, NVLink peer-to-peer prefetch overhead	DWDP improves end-to-end output TPS/GPU by 8.8% at 20-100 TPS/user (1.01-1.10x TPS/GPU and 1.07-1.16x TTFT across token-budget settings) by removing collective synchronization and prefetching MoE weights over NVLink.	Requires high-bandwidth all-to-all connectivity (NVL72-class); gains not monotonic in input length; copy-engine contention with compute kernels must be managed.	high	full-text	Replaces layer-wise all-to-all synchronization with async NVLink weight prefetch in MoE inference, quantifying interconnect-centric scheduling on NVL72.
k2026provisio	field study	Meta 150MW AI datacenter with 83K GB200 GPUs; end-to-end power management across planning, deployment, and runtime phases	cluster throughput, performance-per-watt, power limits, per-GPU RDMA bandwidth	The cluster doubles scale-out RDMA bandwidth per GPU vs the reference design via two 400Gbps CX7 NICs per Grace CPU, and a ~1000W GPU power limit (vs 1200W) improves total cluster throughput by ~9% (GB200 delivering 2.4x per-GPU performance of H100 at 960W).	Single-operator (Meta) production experience; power-centric framing; numbers tied to GB200-era hardware and specific workloads.	high	full-text	Production-scale datapoint on AI-cluster networking (2x400Gbps RDMA per GPU) and power/throughput trade-offs for 100MW-class LLM training clusters.
aashaka2023taccl	systems paper	TACCL collective synthesis for DGX-2 and NDv2 topologies (PCIe Gen3, NVLink, InfiniBand NICs); Allgather/AlltoAll/AllReduce; Transformer-XL and BERT training	collective time vs NCCL, end-to-end training speedup, synthesis time	TACCL-synthesized algorithms outperform NCCL by up to 6.7x (AllReduce up to 28% better for <=1MB buffers on NDv2) and speed up end-to-end Transformer-XL/BERT training by 11%-2.3x.	Synthesis can be slow (up to ~1800s for some sketches); requires designer-provided communication sketches; topology-specific results.	high	full-text	Automated synthesis of topology-aware collective algorithms (respecting PCIe/NVLink/IB link constraints), a key technique for optimizing all-to-all/allreduce in LLM training.
hao2023attentio	systems paper	Ring Attention on 8x A100 NVLink, 32x A100 InfiniBand, and TPUv3-512; 3B/7B/13B models; up to millions of tokens context	max context length, model FLOPs utilization (MFU), throughput, communication-computation overlap condition	Ring Attention trains/infers sequences up to device-count times longer (e.g., 256x on TPUv3-512 for 13B) by overlapping KV-block communication with attention compute, requiring block size >= FLOPs/bandwidth ratio (F/B).	Overlap requires sufficient bandwidth (e.g., with 186GB/s and 196 TFLOPS an MFU of ~40% is the break-even); ppermute-based ring communication only between adjacent hosts.	high	full-text	Foundational ring-attention result showing KV communication fully hidden behind compute; defines the bandwidth-to-FLOPS ratio governing long-context scaling.
amy2024context	systems paper	Context-parallel prefill on up to 128 H100 GPUs across 16 nodes (Grand Teton: 8x H100 NVLink per host, 96GB HBM2e, 2.4TB/s); inter-node RDMA and TCP; Llama3 405B	prefill latency (TTFT), parallelization efficiency, FLOPS utilization, KV cache capacity	Achieves 1M-token prefill of Llama3 405B in 77s (93% parallelization efficiency, 63% FLOPS utilization) and 128K prefill in 3.8s, with near-linear scaling on both RDMA- and TCP-interconnected hosts.	Power-limited H100s (500W, 2.4TB/s HBM) reduce peak FLOPS vs stock H100; All2All latency not in base heuristics (refined in appendix); decode-phase gains smaller.	high	full-text	Shows ring-based context parallelism scales near-linearly to 128 GPUs even on medium/low-bandwidth interconnects, key evidence for long-context inference over commodity fabrics.
mohammad2020megatron	systems paper	Megatron-LM tensor parallelism; up to 8.3B-parameter transformer (GPT-2-like) on 512 GPUs (DGX-2H class)	sustained petaFLOPs, scaling efficiency, FLOPS fraction of peak, communication ops per layer	Megatron-LM sustains 15.1 PetaFLOPs across 512 GPUs (76% scaling efficiency vs a 39-TFLOP single-GPU baseline at 30% of peak) using only 2 all-reduces forward and 2 backward per transformer layer.	Intra-layer parallelism only; requires high-bandwidth intra-node interconnects for the all-reduce-heavy scheme; data-parallel groups add inter-node traffic.	high	full-text	Seminal tensor-parallel design minimizing allreduce count per layer; the communication baseline against which later interconnect-aware LLM training systems are measured.
braun2020analyz	benchmark	InfiniBand EDR 100Gbps vs HDR 200Gbps over PCIe Gen3 with HPC, Cloud, and Deep Learning workloads	application-level performance across interconnect generations (bandwidth, latency, DL training throughput)	The latest InfiniBand HDR interconnect delivers the best performance across all three computing domains (specific speedup numbers not reported in abstract).	Abstract-only; PCIe Gen3 host interface may bottleneck newer NICs; results tied to 2020-era hardware and middleware.	moderate	abstract-only	Quantifies end-user benefit of moving from InfiniBand EDR to HDR generations, grounding expectations for RDMA fabric upgrades in DL training.

Swipe sideways to see all columns.

References

Gangidi, Adithya et al. (2024). RDMA over Ethernet for Distributed Training at Meta Scale — arXiv preprint. Abstract only. Production evidence that RoCE/Ethernet is viable as the interconnect fabric for large-scale AI training.doi:10.1145/3651890.3672233
Batlle Casellas, Alex et al. (2025). Scaling LLM Training Using RDMA over Converged Ethernet — Proceedings of the SC '25 Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis. Abstract only. Head-to-head benchmark of RoCE vs InfiniBand for LLM training, supporting cost-effective Ethernet-based interconnects.doi:10.1145/3731599.3767453
Hoefler, Torsten et al. (2023). Datacenter Ethernet and RDMA: Issues at Hyperscale — arXiv preprint. Full text read. Foundational critique of RoCE/PFC (headroom buffers, victim flows, go-back-N, ECN) that motivates Ultra Ethernet for AI datacenters.doi:10.48550/arxiv.2302.03337
Khan, Tarannum et al. (2022). Impact of RoCE Congestion Control Policies on Distributed Training of DNNs — arXiv preprint. Full text read. Shows general-datacenter congestion control does not transfer to collective-based (All-Reduce/All-To-All) training networks, guiding interconnect CC design.doi:10.48550/arxiv.2207.10898
He, Xin et al. (2025). Rate-adaptive RDMA congestion control for AI clusters — Journal of Cloud Computing. Abstract only. RDMA congestion control optimization for latency-sensitive AI applications on lossless fabrics.doi:10.1186/s13677-025-00830-0
Li, Wenxue et al. (2025). Revisiting RDMA Reliability for Lossy Fabrics — arXiv preprint. Abstract only. Enables efficient RDMA over lossy (non-PFC) Ethernet fabrics, relevant to RoCE scalability for LLM training.doi:10.1145/3718958.3750480
Khalilov, Mikhail et al. (2025). SDR-RDMA: Software-Defined Reliability Architecture for Planetary Scale RDMA Communication — arXiv preprint. Full text read. Software-defined reliability stack for planetary-scale RDMA, key for multi-datacenter LLM training collectives.doi:10.48550/arxiv.2505.05366
Lin, Shengkai et al. (2025). SHIFT: Exploring the Boundary of RDMA Network Fault Tolerance — arXiv preprint. Full text read. Extends RDMA fault tolerance to cross-NIC level and quantifies network failures' share in large-scale LLM training.doi:10.48550/arxiv.2512.11094
Hua, Xiaoxiang et al. (2026). Communication-Semantic-Aware RDMA Loss Recovery for QP-scalable Hyperscale AI Training — arXiv preprint. Full text read. Addresses QP scalability and tail latency of RDMA collectives (All-Reduce/All-to-All) for trillion-parameter training.doi:10.48550/arxiv.2606.20582
Yang, Zhiyi et al. (2025). DCTS-RDMA: Adaptive FEC via Dynamic Coding for Efficient RDMA over Lossy Networks — Lecture Notes in Computer Science. Abstract only. FEC-based alternative to lossless fabrics for RDMA, relevant to RoCE reliability design.doi:10.1007/978-3-032-10459-5_19
Snyder, John et al. (2023). RDMA Congestion Control: It Is Only for the Compliant — IEEE Micro. Abstract only. Security/isolation angle on RDMA congestion control for shared AI clouds.doi:10.1109/mm.2022.3208746
Wan, Zirui et al. (2025). RHCC: Revisiting Intra-Host Congestion Control in RDMA Networks — IEEE Transactions on Networking. Abstract only. Highlights intra-host congestion as an emerging bottleneck as RNIC speeds outpace host interconnects.doi:10.1109/ton.2024.3524247
Shan, Danfeng et al. (2026). Efficient Headroom Allocation With Two-Level Flow Control for Lossless Datacenter Networks — IEEE Transactions on Networking. Abstract only. Headroom buffer inefficiency in lossless Ethernet, a core cost of PFC-based RoCE deployments.doi:10.1109/ton.2025.3596437
Liu, Jingling et al. (2026). FAR: Fast and Accurate Rate Control for Lossless Datacenter Networks — IEEE Transactions on Networking. Abstract only. Combines flow pausing and congestion control for low-latency datacenter (incl. RDMA) networks.doi:10.1109/ton.2026.3655983
Planeta, Maksym et al. (2025). CoRD: Converged RDMA Dataplane — 2025 IEEE International Parallel and Distributed Processing Symposium (IPDPS). Abstract only. Challenges the kernel-bypass assumption of RDMA stacks, relevant to cloud-hosted RDMA AI training.doi:10.1109/ipdps64566.2025.00099
Li, Wenxue et al. (2023). Gleam: An RDMA-accelerated Multicast Protocol for Datacenter Networks — arXiv preprint. Full text read. Brings reliable multicast to RDMA RC transport, a building block for group communication in training frameworks.doi:10.48550/arxiv.2307.14074
Tian, F. et al. (2021). Accelerating Distributed Deep Learning using Multi-Path RDMA in Data Center Networks — arXiv preprint. Abstract only. Software multi-path load balancing for RDMA, addressing the single-path limitation of RNIC hardware.doi:10.1145/3482898.3483363
Luo, Changyun et al. (2024). FlowStar: Fast Convergence Per-Flow State Accurate Congestion Control for InfiniBand — IEEE/ACM Transactions on Networking. Abstract only. InfiniBand congestion control addressing victim flows and slow-convergence congestion spreading.doi:10.1109/tnet.2024.3363658
Cascajo, Alberto et al. (2023). Monitoring InfiniBand Networks to React Efficiently to Congestion — IEEE Micro. Abstract only. Operational tuning of InfiniBand congestion control parameters in real HPC clusters.doi:10.1109/mm.2023.3241840
Maglione-Mathey, German et al. (2020). Path2SL: Leveraging InfiniBand Resources to Reduce Head-of-Line Blocking in Fat Trees — IEEE Micro. Abstract only. InfiniBand queuing/Virtual Lane mechanisms for congestion mitigation in HPC interconnects.doi:10.1109/mm.2019.2949280
Katebzadeh, M. R. Siavash et al. (2020). Evaluation of an InfiniBand Switch: Choose Latency or Bandwidth, but Not Both — arXiv preprint. Abstract only. Measurement methodology (RPerf) and evidence on InfiniBand switch behavior under mixed workloads.doi:10.1109/ispass48437.2020.00033
Luo, Changyun et al. (2024). Alarm: An Adaptive Routing Algorithm Based on One-Way Delay for Infiniband — IEEE Transactions on Network Science and Engineering. Abstract only. Adaptive routing to balance traffic in InfiniBand-dominated HPC/AI clusters.doi:10.1109/tnse.2024.3382295
Wei, Ying et al. (2023). 9.3 NVLink-C2C: A Coherent Off Package Chip-to-Chip Interconnect with 40Gbps/pin Single-ended Signaling — arXiv preprint. Abstract only. Documents NVLink-C2C as the high-bandwidth CPU-GPU interconnect enabling unified memory for AI/HPC.doi:10.1109/isscc42615.2023.10067395
Ishii, Alexander T. & Wells, Ryan S. (2022). The Nvlink-Network Switch: Nvidia’s Switch Chip for High Communication-Bandwidth Superpods — arXiv preprint. Abstract only. Historical context source for NVLink; minimal extractable evidence.doi:10.1109/hcs55958.2022.9895480
Li, Ang et al. (2020). Evaluating Modern GPU Interconnect: PCIe, NVLink, NV-SLI, NVSwitch and GPUDirect — IEEE Transactions on Parallel and Distributed Systems. Abstract only. Empirical characterization of intra-node GPU interconnects (PCIe/NVLink/NVSwitch) foundational to multi-GPU training.doi:10.1109/tpds.2019.2928289
Shen, Ao et al. (2025). FlexLink: Boosting your NVLink Bandwidth by 27% without accuracy concern — arXiv preprint. Full text read. Directly addresses intra-node interconnect utilization (NVLink+PCIe+RDMA aggregation) for LLM training/inference collectives.doi:10.48550/arxiv.2510.15882
Werner, Felix et al. (2025). Towards Memory Disaggregation via NVLink C2C: Benchmarking CPU-Requested GPU Memory Access — arXiv preprint. Abstract only. Quantifies NVLink-C2C viability for memory expansion, informing memory-disaggregation interconnect choices.doi:10.1145/3723851.3723853
Vemuri, Sai Krishna et al. (2025). Communication-Centric UALink and NVLink Comparison for Large Scale AI Training — 2025 IEEE 32nd International Conference on High Performance Computing, Data and Analytics Workshop (HiPCW). Abstract only. Compares open UALink vs proprietary NVLink for AI-training collectives, informing interconnect standardization.doi:10.1109/hipcw66559.2025.00020
Choi, Yea Rem & Stegailov, Vladimir (2022). Multi-GPU GEMM Algorithm Performance Analysis for Nvidia and AMD GPUs Connected by NVLink and PCIe — Communications in computer and information science. Abstract only. Intra-node NVLink/NVSwitch performance analysis for multi-GPU compute kernels.doi:10.1007/978-3-031-24145-1_23
Luo, Shutian et al. (2026). C2CServe: Leveraging NVLink-C2C for Elastic Serverless LLM Serving on MIG — arXiv preprint. Full text read. Shows NVLink-C2C bandwidth reshaping LLM inference/serving architecture, an interconnect-centric serving system.doi:10.48550/arxiv.2605.19481
Zhang, Yicheng et al. (2025). NVBleed: Covert and Side-Channel Attacks on NVIDIA Multi-GPU Interconnect — arXiv preprint. Full text read. Security analysis of the NVLink interconnect in multi-GPU and cloud AI systems, documenting leakage via contention and counters.doi:10.48550/arxiv.2503.17847
Zhang, Yicheng et al. (2024). Beyond the Bridge: Contention-Based Covert and Side Channel Attacks on Multi-GPU Interconnect — arXiv preprint. Abstract only. Demonstrates NVLink congestion leakage as a side channel in multi-GPU systems.doi:10.1109/seed61283.2024.00014
Singhania, Prajwal et al. (2025). Understanding and Improving Communication Performance in Multi-node LLM Inference — arXiv preprint. Full text read. Shows all-reduce/inter-node interconnect (Slingshot, InfiniBand) as the key bottleneck in multi-node LLM inference.doi:10.48550/arxiv.2511.09557
Hoefler, Torsten et al. (2025). Ultra Ethernet's Design Principles and Architectural Innovations — arXiv preprint. Full text read. The Ethernet ecosystem's answer to InfiniBand for AI/HPC: defines the modernized high-performance Ethernet transport.doi:10.48550/arxiv.2508.08906
Metz, Joachim (2024). Empowering AI Workloads in Ultra Ethernet Consortium — arXiv preprint. Abstract only. Motivation and consortium context for Ultra Ethernet as the AI interconnect standard.doi:10.1109/sum60964.2024.10614558
Arsid, Rajesh (2026). UALink, Ultra Ethernet, and PCIe: Transforming Next-Generation HPC and AI Workloads — Journal of Information Systems Engineering & Management. Abstract only. Positions UALink/Ultra Ethernet/CXL as the emerging hierarchical interconnect stack for scale-out AI, directly relevant to RDMA/Ethernet/RoCE and scale-up fabrics for LLM training clusters.doi:10.52783/jisem.v11i2s.14455
Alladi, Vyshnavi & Sass, Ronald (2026). Impact of Ultra Ethernet Transport on Co-Tenant Latency in Reconfigurable Data Center NICs: A Simulation Study — arXiv preprint. Abstract only. Supports incremental adoption of Ultra Ethernet (RoCE-class) transport sharing NICs with legacy tenants in hyperscaler clouds for AI/HPC traffic.doi:10.1109/fccm68464.2026.00080
Hu, Zhiyi et al. (2025). Demystifying NCCL: An In-depth Analysis of GPU Communication Protocols and Algorithms — arXiv preprint. Full text read. Foundational reference demystifying NCCL internals (protocol variants, GPUDirect RDMA transports, ring/tree algorithms) that underpin large-scale LLM training interconnect stacks.doi:10.48550/arxiv.2507.04786
Weingram, Adam et al. (2023). xCCL: A Survey of Industry-Led Collective Communication Libraries for Deep Learning — Journal of Computer Science and Technology. Abstract only. Maps the collective communication library landscape (NCCL/RCCL/MSCCL/ACCL/Gloo) relevant to interconnect-aware optimization for deep learning workloads.doi:10.1007/s11390-023-2894-6
Cowan, Meghan et al. (2023). MSCCLang: Microsoft Collective Communication Language — arXiv preprint. Abstract only. Enables topology-aware custom collectives (e.g., for NVLink/InfiniBand topologies), an approach later built upon by synthesis-based schedulers such as TACCL and Canvas.doi:10.1145/3575693.3575724
Hei, Chenyang et al. (2025). Canvas: Scalable Collective Communication Scheduling for Large-Scale GPU Clusters — 2025 IEEE 33rd International Conference on Network Protocols (ICNP). Abstract only. Advances scalable, near-optimal topology-aware collective scheduling to overcome vendor-library bandwidth bottlenecks in large GPU clusters used for LLM training.doi:10.1109/icnp65844.2025.11192367
Cao, Jiamin et al. (2024). Crux: GPU-Efficient Communication Scheduling for Deep Learning Training — Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication. Abstract only. Shows communication contention among co-located DLT jobs wastes GPU compute and that prioritizing high-intensity flows mitigates it in shared training clusters.doi:10.1145/3651890.3672239
Huang, Jiajun et al. (2023). gZCCL: Compression-Accelerated Collective Communication Framework for GPU Clusters — arXiv preprint. Full text read. Demonstrates accuracy-aware lossy compression inside GPU-aware collectives as a way to stretch limited inter-node bandwidth (100 Gbps-class) in large-scale training.doi:10.48550/arxiv.2308.05199
Li, Baojia et al. (2024). TCCL: Co-optimizing Collective Communication and Traffic Routing for GPU-centric Clusters — arXiv preprint. Abstract only. Field evidence that GPU-centric networks (>10K GPUs) require co-design of collective communication and centralized routing to avoid congestion and jitter in LLM training.doi:10.1145/3672198.3673799
Liu, Yipeng et al. (2026). UBEP: Re-architecting Expert Parallelism Communication Library for Production Superpods — SIGCOMM 2026. Full text read. Quantifies how BSP serialization, synchronization overhead, and distance-agnostic scheduling waste high-bandwidth scale-up fabrics (UB/NVLink-class) in MoE all-to-all traffic.doi:10.48550/arxiv.2607.06202
Goldman, Amos et al. (2026). NCCL EP: Towards a Unified Expert Parallel Communication API for NCCL — arXiv preprint. Full text read. Brings MoE dispatch/combine (fine-grained all-to-all over RDMA+NVLink) natively into NCCL, unifying device-initiated expert-parallel communication for training and inference.doi:10.48550/arxiv.2603.13606
Zheng, Yusheng (2026). NCCLbpf: Verified, Composable Policy Execution for GPU Collective Communication — arXiv preprint. Full text read. Demonstrates verified, composable, hot-reloadable policy extension (eBPF-style) for NCCL's plugin interface, improving collective throughput without modifying NCCL source.doi:10.48550/arxiv.2603.11438
Jia, Ziyang et al. (2026). NIXT: A NCCL Inspector Exporter Tool for Observability of Collective Communication in Large Model Training — arXiv preprint. Full text read. Provides production-scale observability into collective communication (bandwidth, stragglers, phase attribution) needed to diagnose interconnect bottlenecks in LLM pretraining.doi:10.48550/arxiv.2608.01449
Shen, Siyuan et al. (2026). Every Microsecond Matters: Achieving Near Speed-of-Light Latency in GPU Collectives — arXiv preprint. Full text read. Shows latency, not just bandwidth, dominates small collectives on decode-heavy LLM inference critical paths, and quantifies microsecond-level gains achievable on scale-up interconnects.doi:10.48550/arxiv.2607.16100
Singh, Siddharth et al. (2025). The Big Send-off: Scalable and Performant Collectives for Deep Learning — arXiv preprint. Full text read. Quantifies large headroom in vendor collective libraries (RCCL/NCCL) at supercomputer scale, informing interconnect-aware collective design for large-buffer DL workloads.doi:10.48550/arxiv.2504.18658
Zhang, Mingjun et al. (2025). An Efficient, Reliable and Observable Collective Communication Library in Large-scale GPU Training Clusters — arXiv preprint. Full text read. Production evidence that SM competition, RNIC port failures, and transient RDMA anomalies waste GPU resources, and that SM-free P2P plus fault-tolerant QPs recover most of the loss.doi:10.48550/arxiv.2510.00991
Zhang, Lin et al. (2023). DeAR: Accelerating Distributed Deep Learning with Fine-Grained All-Reduce Pipelining — arXiv preprint. Full text read. Shows fine-grained overlap of decoupled all-reduce with both backward and forward passes, with larger gains on low-bandwidth/high-latency Ethernet than on InfiniBand.doi:10.48550/arxiv.2302.12445
Wang, Guanhua et al. (2023). ZeRO++: Extremely Efficient Collective Communication for Giant Model Training — arXiv (Cornell University). Full text read. Communication-volume reduction techniques that make ZeRO-3 viable on bandwidth-limited Ethernet clusters, a key regime for LLM training outside InfiniBand/NVLink datacenters.doi:10.48550/arxiv.2306.10209
Park, Gyeongseo et al. (2026). FCDP: Fully Cached Data Parallel for Communication-Avoiding Large-Scale Training — arXiv.org. Abstract only. Reframes host memory as a fast caching layer to bypass slow inter-node interconnects, a practical trick for LLM training on non-NVLink/InfiniBand commodity fabrics.doi:10.48550/arxiv.2602.06499
Xu, Guanbin et al. (2026). Lagom: Unleashing the Power of Communication and Computation Overlapping for Distributed LLM Training — arXiv preprint. Full text read. Shows communication tuning must also target computation-bottlenecked regimes, informing overlap strategies on both RDMA and NVLink fabrics in LLM training.doi:10.48550/arxiv.2602.20656
Jia, Ziyang et al. (2024). PCCL: Energy-Efficient LLM Training with Power-Aware Collective Communication — 2024 IEEE 42nd International Conference on Computer Design (ICCD). Abstract only. Adds an energy dimension to interconnect/collective design: communication kernels dominate training time and offer DVFS-based power savings with minimal performance cost.doi:10.1109/iccd63220.2024.00023
Jin, Xin et al. (2024). SDCC: software-defined collective communication for distributed training — Science China Information Sciences. Abstract only. Software-defined, transport-agnostic collectives (including kernel-bypass/RDMA-class transports) as an alternative to tightly coupled vendor communication stacks.doi:10.1007/s11432-023-3894-4
Li, Wenxue et al. (2024). Understanding Communication Characteristics of Distributed Training — arXiv preprint. Abstract only. Empirical basis for predicting communication overhead in distributed LLM training, informing when interconnect optimizations (RDMA, collectives) matter most.doi:10.1145/3663408.3663409
Samsi, Siddharth et al. (2020). Benchmarking network fabrics for data distributed training of deep neural networks — arXiv preprint. Abstract only. Benchmark evidence that for modest data-parallel workloads interconnect choice (Ethernet vs OmniPath) matters little, contrasting with today's bandwidth-hungry LLM training.doi:10.1109/hpec43674.2020.9286232
Chen, Peiqing et al. (2026). Don't Let a Few Network Failures Slow the Entire AllReduce — arXiv preprint. Full text read. Quantifies and nearly eliminates the AllReduce penalty of NIC-failure rerouting (e.g., PXN-style failover) that otherwise throttles entire data-parallel training collectives.doi:10.48550/arxiv.2606.01680
Wang, Wei et al. (2025). Reliable and Resilient Collective Communication Library for LLM Training and Serving — arXiv.org. Abstract only. Fault-tolerant collective library exploiting multi-NIC RDMA connection migration to prevent job-killing timeouts and checkpoint rollbacks in InfiniBand-based training clusters.doi:10.48550/arxiv.2512.25059
Pan, Lichen et al. (2025). Comprehensive Deadlock Prevention for GPU Collective Communication — Proceedings of the Twentieth European Conference on Computer Systems. Abstract only. Catalogues correctness hazards (deadlocks) in collective communication stacks that practitioners must handle when orchestrating interconnect-heavy LLM training.doi:10.1145/3689031.3717466
Arzani, Behnaz et al. (2023). Rethinking Machine Learning Collective Communication as a Multi-Commodity Flow Problem — arXiv (Cornell University). Full text read. Recasts ML collective scheduling as a traffic-engineering-style flow problem, producing scalable, higher-quality schedules for AllToAll/AllGather/AllReduce on GPU fabrics.doi:10.48550/arxiv.2305.13479
Hamidouche, Khaled et al. (2025). GPU-Initiated Networking for NCCL — arXiv preprint. Full text read. Documents the hardware/software path for GPU-initiated RDMA (GPUDirect Async / DOCA GPUNetIO), central to low-latency MoE all-to-all on InfiniBand/RoCE fabrics.doi:10.48550/arxiv.2511.15076
Hwang, Chang Ho et al. (2022). Tutel: Adaptive Mixture-of-Experts at Scale — arXiv (Cornell University). Full text read. Adaptive parallelism and hierarchical all-to-all for MoE, addressing dynamic token-routing traffic patterns on A100 clusters relevant to LLM-scale expert parallelism.doi:10.48550/arxiv.2206.03382
Rajbhandari, Samyam et al. (2022). DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale — arXiv (Cornell University). Full text read. Establishes MoE all-to-all communication as a first-class cost in large-model training/inference systems, motivating interconnect- and RDMA-aware expert-parallel design.doi:10.48550/arxiv.2201.05596
DeepSeek-AI et al. (2024). DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model — arXiv (Cornell University). Full text read. Production MoE architecture whose device-limited routing and communication-balance losses directly shape all-to-all traffic on GPU interconnects during training and inference.doi:10.48550/arxiv.2405.04434
Dai, Damai et al. (2024). DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models — arXiv preprint. Abstract only. Citation-only record of the DeepSeekMoE architecture paper; relevant as MoE routing/expert-specialization context for all-to-all communication in LLM training.doi:10.18653/v1/2024.acl-long.70
DeepSeek-AI et al. (2024). DeepSeek-V3 Technical Report — arXiv (Cornell University). Full text read. Flagship evidence that computation-communication overlap plus IB/NVLink-optimized all-to-all kernels make 671B-scale MoE training communication-near-free and cost-effective.doi:10.48550/arxiv.2412.19437
Nie, Xiaonan et al. (2024). LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing — arXiv (Cornell University). Full text read. Quantifies all-to-all as the dominant cost in MoE training (45%+ of time on 200 Gb/s fabrics) and shows token-similarity compression as a communication-volume mitigation.doi:10.48550/arxiv.2411.08446
Zhang, Zhonghao et al. (2026). HALO: Hiding All-to-all via Locality-aware Overlap for Efficient MoE Training — arXiv preprint. Abstract only. Shows all-to-all communication in expert-parallel MoE training can be hidden via locality-aware overlap and shadow experts, a key technique for interconnect-bound LLM training.doi:10.2139/ssrn.6913743
Qi, Shuyao et al. (2026). FEPLB: Exploiting Copy Engines for Nearly Free MoE Load Balancing in Distributed Training — arXiv preprint. Full text read. Exploits the NVLink Copy Engine as a near-free intra-node communication channel parallel to compute kernels, informing NVLink usage for MoE load balancing in LLM training.doi:10.48550/arxiv.2604.19654
Qin, Huihuang et al. (2026). Reducing Cross-Pod Communication Overhead for MoE Model Training With Hybrid Parallelism in Multi-Tenant Clusters — IEEE Transactions on Parallel and Distributed Systems. Abstract only. Addresses cross-Pod placement to mitigate intra-/inter-Pod bandwidth and latency discrepancies for MoE training, relevant to interconnect-aware job placement.doi:10.1109/tpds.2026.3668417
Zhang, Qijun et al. (2026). Accelerating MoE with Dynamic In-Switch Computing on Multi-GPUs — arXiv preprint. Full text read. Extends NVLink SHARP in-switch reduction to irregular dynamic MoE traffic, showing in-switch computing can cut redundant dispatch/combine transfers in LLM training.doi:10.48550/arxiv.2605.05607
Zhu, Ruidong et al. (2025). MegaScale-Infer: Efficient Mixture-of-Experts Model Serving with Disaggregated Expert Parallelism — arXiv preprint. Abstract only. Partial abstract frames memory-bound MoE inference efficiency, relevant to memory/interconnect pressure in LLM serving.doi:10.1145/3718958.3750506
Cai, Weilin et al. (2024). Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Experts — arXiv (Cornell University). Full text read. Quantifies how NVLink bandwidth (A30 vs A800) changes All-to-All overhead and shows shortcut-connected MoE decouples communication from computation for overlap.doi:10.48550/arxiv.2404.05019
Wang, Weiyang et al. (2022). TopoOpt: Co-optimizing Network Topology and Parallelization Strategy for Distributed Training Jobs — arXiv preprint. Full text read. Pioneers co-optimizing network topology with AllReduce traffic mutability for DNN training, foundational for topology-aware interconnect design (RDMA/OCS).doi:10.48550/arxiv.2202.00433
Namkoong, Seok et al. (2025). SortingHat: System Topology-aware Scheduling of Deep Neural Network Models on Multi-GPU Systems — Proceedings of the 39th ACM International Conference on Supercomputing. Abstract only. Shows multi-GPU LLM inference gains depend on co-scheduling data-transfer parallelism with system topology, relevant to interconnect-aware scheduling.doi:10.1145/3721145.3728490
Chang, Zihan et al. (2026). SpiderFlow: Efficient Topology-Aware Scheduling for LLM Training Across Decentralized GPU Clusters — Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Abstract only. Citation-only record (SpiderFL, ACL 2026); likely federated-learning communication/scheduling work but no extractable interconnect evidence.doi:10.18653/v1/2026.acl-long.619
Ye, Zhisheng et al. (2023). Deep Learning Workload Scheduling in GPU Datacenters: A Survey — ACM Computing Surveys. Abstract only. Provides a taxonomy of GPU-datacenter scheduling including communication- and resource-aware methods, context for interconnect-aware training scheduling.doi:10.1145/3638757
Yu, Menglu et al. (2022). On scheduling ring-all-reduce learning jobs in multi-tenant GPU clusters with communication contention — arXiv preprint. Abstract only. Models communication contention between co-located all-reduce training jobs, relevant to RDMA/collective scheduling in GPU clusters.doi:10.1145/3492866.3549716
Dong, Fangxiao et al. (2026). Training-Phase-Aware Optical Circuit Switching Reconfiguration for Large Language Model — Optical Fiber Communication Conference (OFC) 2026. Abstract only. Evidence that reconfigurable optical (OCS) topologies can accelerate LLM training collectives by matching topology to the communication phase.doi:10.1364/ofc.2026.m3f.5
Cho, Sanghun et al. (2023). Logical/Physical Topology-Aware Collective Communication in Deep Learning Training — 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA). Abstract only. Demonstrates logical/physical topology-aware AllReduce with computation chaining on an NVLink-based DGX-1, relevant to collective optimization.doi:10.1109/hpca56546.2023.10071117
Won, William et al. (2023). ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale — arXiv preprint. Abstract only. Provides an open simulator for interconnect/topology-aware distributed training design-space exploration (NVLink, InfiniBand, collectives, disaggregated memory).doi:10.1109/ispass57527.2023.00035
Jiang, Ziheng et al. (2024). MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs — arXiv (Cornell University). Full text read. Production evidence that communication-computation overlap, network tuning, and observability sustain 55% MFU at 12k+ GPUs - a reference point for interconnect-bound LLM training.doi:10.48550/arxiv.2402.15627
Meng, Qingkai et al. (2025). Astral: A Datacenter Infrastructure for Large Language Model Training at Scale — arXiv preprint. Abstract only. Field experience with same-rail network design for LLM training clusters, relevant to datacenter interconnect architecture.doi:10.1145/3718958.3750521
Jouppi, Norman P. et al. (2023). TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings — arXiv preprint. Abstract only. Seminal evidence that OCS-based reconfigurable optical interconnects are cheaper and lower power than InfiniBand and sustain ~60% peak FLOPS for LLM training.doi:10.1145/3579371.3589350
Liu, Hong et al. (2023). Lightwave Fabrics: At-Scale Optical Circuit Switching for Datacenter and Machine Learning Systems — arXiv preprint. Abstract only. Shows reconfigurable optical lightwave fabrics improve availability and LLM training performance, supporting OCS-based interconnect arguments.doi:10.1145/3603269.3604836
Shou, Chenchen et al. (2025). InfiniteHBD: Building Datacenter-Scale High-Bandwidth Domain for LLM with Optical Circuit Switching Transceivers — arXiv preprint. Abstract only. Frames the high-bandwidth-domain (NVLink-domain) scalability/cost/fault tradeoff space for LLM training interconnects.doi:10.1145/3718958.3750468
Feng, Shi et al. (2025). Mercury: A Reconfigurable Datacenter Network with Collaborative Optical Timeslot Switching and Optical Circuit Switching — Optical Fiber Communication Conference (OFC) 2025. Abstract only. Evidence for nanosecond-scale reconfigurable optical switching as an interconnect option for ML training.doi:10.1364/ofc.2025.m4h.3
Yang, Chongyi et al. (2025). BEAM: A Multi-Channel Optical Interconnect for Multi-GPU Systems — 2025 Design, Automation &amp; Test in Europe Conference (DATE). Abstract only. Argues for in-package optical (vs electrical) GPU interconnects for bandwidth and power at scale, relevant to NVLink-alternative roadmaps.doi:10.23919/date64628.2025.10993197
Zhou, Linjie & Luo, Xianshu (2026). Co-packaged optics: enabling the next generation high-bandwidth and energy-efficient interconnects — Advanced Photonics Nexus. Abstract only. Points to co-packaged optics as a packaging-level interconnect direction for AI/network systems.doi:10.1117/1.apn.5.3.030101
Nabavinejad, Seyed Morteza et al. (2020). An Overview of Efficient Interconnection Networks for Deep Neural Network Accelerators — IEEE Journal on Emerging and Selected Topics in Circuits and Systems. Abstract only. Background survey on on-chip NoC/interconnect design tradeoffs for DNN accelerators, upstream of scale-out interconnects like NVLink/RoCE.doi:10.1109/jetcas.2020.3022920
Jung, Myoungsoo (2025). Compute Can't Handle the Truth: Why Communication Tax Prioritizes Memory and Interconnects in Modern AI Infrastructure — arXiv preprint. Full text read. Useful synthesis of the communication-tax (35-70% of training time) and the NVLink/CXL/UALink bandwidth hierarchy for LLM infrastructure.doi:10.48550/arxiv.2507.07223
Gouk, Donghyun et al. (2023). Memory Pooling With CXL — IEEE Micro. Abstract only. Quantifies CXL.mem vs RDMA for memory disaggregation, relevant to KV-cache/memory pooling for LLM serving.doi:10.1109/mm.2023.3237491
Gu, Yufeng et al. (2025). PIM Is All You Need: A CXL-Enabled GPU-Free System for Large Language Model Inference — arXiv preprint. Abstract only. Frames memory-bandwidth-bound LLM inference as the driver for high-bandwidth memory and interconnect systems.doi:10.1145/3676641.3716267
Ko, Younghoon et al. (2026). IBEX: Internal Bandwidth‑Efficient Compression Architecture for Scalable CXL Memory Expansion — Proceedings of the 40th ACM International Conference on Supercomputing. Abstract only. Addresses bandwidth/capacity limits of CXL memory expanders, an emerging tier for LLM KV-cache offload.doi:10.1145/3797905.3800521
Quinn, Derrick et al. (2025). Compute-Enabled CXL Memory Expansion for Efficient Retrieval Augmented Generation — IEEE Micro. Abstract only. Combines CXL memory expansion with near-memory compute for RAG retrieval, relevant to memory-centric LLM inference.doi:10.1109/mm.2025.3575280
Fang, Yunhua et al. (2025). Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System — arXiv preprint. Full text read. Quantifies headroom in KV-cache placement across HBM/off-package DRAM tiers, informing memory-interconnect design for LLM inference.doi:10.48550/arxiv.2508.13231
Wang, Zixuan et al. (2022). Enabling Efficient Large-Scale Deep Learning Training with Cache Coherent Disaggregated Memory Systems — arXiv preprint. Abstract only. Shows cache-coherent interconnects plus disaggregated memory can beat MPI AllReduce for parameter synchronization, relevant to RDMA/CCI tradeoffs.doi:10.1109/hpca53966.2022.00018
Li, Huaicheng et al. (2023). Pond: CXL-Based Memory Pooling Systems for Cloud Platforms — arXiv preprint. Abstract only. Establishes that small CXL pools meet cloud performance goals, informing memory-pooling economics for LLM serving.doi:10.1145/3575693.3578835
Ganjihal, Sanjeev Rao (2026). Topology-Aware Data Movement for Disaggregated GPU Inference — arXiv preprint. Full text read. Quantifies the NVLink/InfiniBand/TCP bandwidth hierarchy (72x spread) that disaggregated inference systems must exploit, critiquing uniform-RDMA designs like DistServe/Splitwise/Mooncake.doi:10.48550/arxiv.2607.28633
Zhong, Yinmin et al. (2024). DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving — arXiv (Cornell University). Full text read. Canonical prefill/decode disaggregation paper showing placement must account for interconnect bandwidth to minimize disaggregation communication.doi:10.48550/arxiv.2401.09670
Patel, Pratyush et al. (2023). Splitwise: Efficient generative LLM inference using phase splitting — arXiv (Cornell University). Full text read. Quantifies throughput/cost gains of phase splitting and the role of fast interconnects (NVLink/InfiniBand) in KV state transfer between phases.doi:10.48550/arxiv.2311.18677
Qin, Ruoyu et al. (2025). Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving — ACM Transactions on Storage. Abstract only. Production proof that disaggregated KVCache using spare CPU/DRAM/SSD/NIC resources scales LLM serving, with data-movement implications across interconnects.doi:10.1145/3773772
Wu, Bingyang et al. (2024). LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism — arXiv (Cornell University). Full text read. Shows elastic sequence parallelism reduces KV-transfer/communication overhead in long-context serving, quantifying the interconnect cost of KV migration across NVLink/InfiniBand.doi:10.48550/arxiv.2404.09526
Liu, Mingxuan et al. (2025). ServerlessPD: Fast RDMA-Codesigned Disaggregated Prefill-Decoding for Serverless Inference of Large Language Models — 2025 IEEE International Conference on Web Services (ICWS). Abstract only. Demonstrates RDMA kernel-integrated remote fork as a mechanism for zero-copy KV state migration in disaggregated serverless LLM serving.doi:10.1109/icws67624.2025.00045
Liu, Yuhan et al. (2025). LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference — arXiv (Cornell University). Full text read. Provides an open-source KV cache layer spanning GPU/CPU/storage/network tiers, quantifying gains of cross-engine KV movement for PD disaggregation and prefix reuse.doi:10.48550/arxiv.2510.09665
Yang, Jiaqi et al. (2026). Beyond Prefill-Decode Disaggregation: Dissecting LLM Inference for Heterogeneous Platforms via Dynamic Operator Scheduling — arXiv (Cornell University). Full text read. Argues PD disaggregation alone is insufficient and models communication/placement costs (all-reduce, gather, scatter) for operator scheduling across heterogeneous interconnects.doi:10.48550/arxiv.2607.25498
Wei, Yuanxin et al. (2024). Communication-Efficient Model Parallelism for Distributed In-Situ Transformer Inference — arXiv preprint. Abstract only. Shows block-parallel inference restructuring to cut communication frequency under weak interconnects, relevant to interconnect-constrained distributed inference.doi:10.23919/date58400.2024.10546617
Li, Zhuohan et al. (2021). TeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language Models — arXiv (Cornell University). Full text read. Exploits token-dimension pipelining to cut communication stalls in model-parallel LLM training; contrasts with allreduce-based Megatron-LM synchronization.doi:10.48550/arxiv.2102.07988
Li, Shenggui et al. (2023). Sequence Parallelism: Long Sequence Training from System Perspective — Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Abstract only. Positional reference for sequence parallelism (SP) as an interconnect-relevant alternative to tensor parallelism for long-sequence LLM training.doi:10.18653/v1/2023.acl-long.134
Brandon, William T. et al. (2023). Striped Attention: Faster Ring Attention for Causal Transformers — arXiv (Cornell University). Full text read. Improves load balance of ring-based context-parallel attention (KV communication overlapped with compute), relevant to long-context training communication patterns.doi:10.48550/arxiv.2311.09431
Gu, Diandian et al. (2024). LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism — arXiv (Cornell University). Full text read. Combines head-parallel and context-parallel attention to use all inter-node NICs (HDR InfiniBand) efficiently, quantifying SeqAlltoAll vs P2P communication trade-offs.doi:10.48550/arxiv.2406.18485
Ghadia, Ravi et al. (2026). Untied Ulysses: Memory-Efficient Context Parallelism via Headwise Chunking — arXiv (Cornell University). Full text read. Shows headwise chunking to shrink all-to-all activation buffers in context-parallel training, addressing the memory side of long-context collective communication.doi:10.48550/arxiv.2602.21196
Jiang, Chenyu et al. (2025). DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelism — Proceedings of the ACM SIGOPS 31st Symposium on Operating Systems Principles. Abstract only. Motivates dynamic (rather than static) context-parallel partitioning to cut unnecessary communication for variable-length long-context training.doi:10.1145/3731569.3764849
Narayanan, Deepak et al. (2021). Efficient large-scale language model training on GPU clusters using megatron-LM — arXiv preprint. Abstract only. Canonical reference on composing parallelisms to reach thousands of GPUs, setting the communication-efficiency baseline that interconnect-aware schedules target.doi:10.1145/3458817.3476209
Wang, Ang et al. (2020). Whale: A Unified Distributed Training Framework. — arXiv (Cornell University). Full text read. Demonstrates graph-level placement of collectives (e.g., NVLink-based AllReduce within nodes) as a lever for efficient giant-model training on heterogeneous clusters.doi:10.48550/arxiv.2011.09208
Sensi, Daniele De et al. (2024). Exploring GPU-to-GPU Communication: Insights into Supercomputer Interconnects — arXiv preprint. Abstract only. Provides comparative interconnect characterization (multi-Tbps intra-node networks) for GPU supercomputers, informing expectations for large-scale LLM training fabrics.doi:10.1109/sc41406.2024.00039
Li, Conglong et al. (2021). 1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speed — arXiv preprint. Full text read. Quantifies allreduce-dominated communication on commodity TCP/Ethernet clusters and shows gradient compression as an interconnect-bandwidth mitigation for LLM-scale training.doi:10.48550/arxiv.2104.06069
Li, Xue et al. (2024). Near-Lossless Gradient Compression for Data-Parallel Distributed DNN Training — arXiv preprint. Abstract only. Near-lossless gradient compression as a way to relieve interconnect bandwidth pressure in data-parallel LLM-scale training.doi:10.1145/3698038.3698541
Li, Yulong et al. (2024). Host-driven In-Network Aggregation on RDMA — IEEE INFOCOM 2024 - IEEE Conference on Computer Communications. Abstract only. Addresses the collision between in-network aggregation (INA) and RDMA reliable connections for DNN training, relevant to RDMA/RoCE-based training clusters.doi:10.1109/infocom52122.2024.10621230
Segal, Raz et al. (2022). Constrained In-network Computing with Low Congestion in Datacenter Networks — IEEE INFOCOM 2022 - IEEE Conference on Computer Communications. Abstract only. Formal foundation for in-network aggregation (e.g., SHARP-style) deployment, underpinning congestion reduction for distributed ML collectives.doi:10.1109/infocom48880.2022.9796980
Zhu, Haowen et al. (2024). When In-Network Computing Meets Distributed Machine Learning — IEEE Network. Abstract only. Survey framing INC (programmable-switch aggregation) as an accelerator for distributed ML, contextualizing switch-offload approaches for collectives.doi:10.1109/mnet.2024.3368138
Wang, Guanhua (2023). Blink: Fast and Generic Collectives for Distributed ML — SIGCOMM 2023. Full text read. Spanning-tree packing over heterogeneous NVLink/PCIe/NIC topologies as an alternative to NCCL ring collectives, directly relevant to collective design for training.doi:10.48550/arxiv.1910.04940
Urata, Ryohei et al. (2022). Mission Apollo: Landing Optical Circuit Switching at Datacenter Scale — arXiv preprint. Abstract only. Production evidence for optical circuit switching in datacenter networks, a candidate technology for future AI-cluster fabric reconfiguration.doi:10.48550/arxiv.2208.10041
Sun, Yan et al. (2023). Demystifying CXL Memory with Genuine CXL-Ready Systems and Devices — arXiv preprint. Full text read. Empirical latency/bandwidth characterization of CXL memory expansion (and CXL vs RDMA remote access), informing memory-disaggregation options for LLM inference.doi:10.48550/arxiv.2303.15375
Das Sharma, Debendra et al. (2023). An Introduction to the Compute Express Link (CXL) Interconnect — arXiv preprint. Abstract only. Reference survey for CXL as an interconnect (memory expansion, pooling, peer-to-peer), a growing alternative/complement to RDMA for disaggregated memory.doi:10.48550/arxiv.2306.11227
Arelakis, Angelos et al. (2024). Streamlining CXL Adoption for Hyperscale Efficiency — arXiv preprint. Full text read. Shows compressed CXL tiers to expand effective memory capacity at hyperscale, addressing memory-capacity constraints in AI inference without extra network traffic.doi:10.48550/arxiv.2404.03551
Agarwal, Saurabh et al. (2024). SYMPHONY: Improving Memory Management for LLM Inference Workloads — arXiv preprint. Full text read. KV-cache migration off the critical path as an alternative to stateful offloading, illustrating state-transfer overheads across cluster interconnects (100Gbps Ethernet).doi:10.48550/arxiv.2412.16434
Li, Wanqian et al. (2026). DWDP: Distributed Weight Data Parallelism for High-Performance LLM Inference on NVL72 — arXiv preprint. Full text read. Replaces layer-wise all-to-all synchronization with async NVLink weight prefetch in MoE inference, quantifying interconnect-centric scheduling on NVL72.doi:10.48550/arxiv.2604.01621
Ardestani, Ehsan K. et al. (2026). Provisioning to Runtime Optimization of a 100 MW-Scale AI Cluster — arXiv preprint. Full text read. Production-scale datapoint on AI-cluster networking (2x400Gbps RDMA per GPU) and power/throughput trade-offs for 100MW-class LLM training clusters.doi:10.48550/arxiv.2605.24461
Shah, Aashaka et al. (2023). TACCL: Guiding Collective Algorithm Synthesis using Communication Sketches — NSDI 2023. Full text read. Automated synthesis of topology-aware collective algorithms (respecting PCIe/NVLink/IB link constraints), a key technique for optimizing all-to-all/allreduce in LLM training.doi:10.48550/arxiv.2111.04867
Liu, Hao et al. (2023). Ring Attention with Blockwise Transformers for Near-Infinite Context — arXiv preprint. Full text read. Foundational ring-attention result showing KV communication fully hidden behind compute; defines the bandwidth-to-FLOPS ratio governing long-context scaling.doi:10.48550/arxiv.2310.01889
Yang, Amy et al. (2024). Context Parallelism for Scalable Million-Token Inference — arXiv preprint. Full text read. Shows ring-based context parallelism scales near-linearly to 128 GPUs even on medium/low-bandwidth interconnects, key evidence for long-context inference over commodity fabrics.doi:10.48550/arxiv.2411.01783
Shoeybi, Mohammad et al. (2020). Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism — NeurIPS 2020. Full text read. Seminal tensor-parallel design minimizing allreduce count per layer; the communication baseline against which later interconnect-aware LLM training systems are measured.doi:10.48550/arxiv.1909.08053
Ruhela, Amit et al. (2020). Analyzing and Understanding the Impact of Interconnect Performance on HPC, Big Data, and Deep Learning Applications: A Case Study with InfiniBand EDR and HDR — 2020 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). Abstract only. Quantifies end-user benefit of moving from InfiniBand EDR to HDR generations, grounding expectations for RDMA fabric upgrades in DL training.doi:10.1109/ipdpsw50202.2020.00147