---
title: "Interconnects for large-scale LLM training and inference: RDMA, InfiniBand, NVLink, and the network stack"
slug: llm-interconnects
question: "What does the evidence from 2020 to 2026 show about how interconnect technologies (RDMA, InfiniBand, NVLink, Ethernet/RoCE, and the software stacks built on them) shape the performance, scaling, and cost of large-scale LLM training and inference?"
status: published
depth: deep
created: 2026-08-08
updated: 2026-08-08
summary: "Large-scale LLM training and inference now depend on a hierarchy of interconnects: NVLink and NVSwitch inside servers, InfiniBand or RoCE Ethernet between them, and collective-communication libraries that turn raw fabric into training progress. The 2020-2026 evidence shows the lossless RDMA fabric that carried early GPT-era training is under strain, with congestion-control pathologies and head-of-line blocking documented in both directions, while production operators engineer around it and a lossy, packet-spraying Ultra Ethernet path is emerging. Meanwhile Mixture-of-Experts models and long-context inference have made all-to-all traffic and KV-cache movement the new communication bottlenecks, and NVLink scale-up domains, UALink, and CXL memory pooling are reshaping where computation and memory live. Confidence is moderate: the transport and collective stories are replicated across many independent studies, but the newest hardware (NVLink 4/5, UALink, GB200-class superpods) is covered mainly by vendor-adjacent or preprint evidence."
disciplines: ["computer systems", "computer networks", "computer architecture"]
tags: ["RDMA", "InfiniBand", "NVLink", "NVSwitch", "RoCE", "Ultra Ethernet", "UALink", "NCCL", "collective communication", "all-to-all", "Mixture-of-Experts", "KV cache", "CXL", "optical circuit switching", "LLM training", "LLM inference"]
source_count: 137
year_range: [2020, 2026]
confidence: moderate
search:
  databases: ["OpenAlex", "Crossref", "arXiv API", "Semantic Scholar (best-effort)"]
  queries:
    - "RDMA"
    - "remote direct memory access"
    - "RDMA congestion control"
    - "RDMA over converged ethernet RoCE"
    - "lossless datacenter network"
    - "InfiniBand"
    - "InfiniBand GPU training"
    - "NVLink"
    - "NVSwitch"
    - "NVLink large language model training"
    - "TACCL topology-aware collective communication"
    - "Blink fast generic collectives distributed machine learning"
    - "Symphony topology-aware placement LLM training"
    - "Astra simulating distributed training topology"
    - "Sirius optical circuit switching distributed machine learning"
    - "Meridian RDMA large language model training"
    - "AliExpress 10,000 GPU training Alibaba"
    - "BytePS unified communication framework"
    - "Ring Attention blockwise transformers"
    - "NCCL"
    - "DeepSpeed-Ulysses sequence parallelism"
    - "Context parallelism long sequence training"
    - "NVLink-Sharp in-network communication acceleration"
    - "SwitchML in-network aggregation"
    - "MSCCL collective communication library"
    - "Hot Box scale-up interconnect"
    - "Tasmanian Devil CXL memory"
    - "eRDMA elastic RDMA cloud"
    - "Jupiter Google network AI"
    - "GB200 NVL72 performance"
    - "collective communication GPU training"
    - "GH200 Grace Hopper interconnect"
    - "Megatron-LM multi-gpu training"
    - "TCP for AI training datacenter"
    - "Whale pipeline parallelism"
    - "Demystifying parallel distributed deep learning survey"
    - "AI datacenter network survey"
    - "lossy ethernet RDMA training"
    - "in-network computing machine learning"
    - "reconfigurable network machine learning training"
    - "GPU networking large language model training"
    - "all-reduce optimization deep learning"
    - "interconnect-aware parallelism LLM"
    - "all-to-all MoE communication"
    - "expert parallelism communication"
    - "topology-aware scheduling GPU training"
    - "network-aware scheduling large model training"
    - "GPU cluster interconnect"
    - "large language model training infrastructure"
    - "datacenter network large language model"
    - "AI training cluster design"
    - "prefill decode disaggregation inference"
    - "KV cache transfer inference"
    - "distributed inference interconnect"
    - "scale-up interconnect AI"
    - "CXL memory expansion AI"
    - "disaggregated memory GPU training"
    - "Ultra Ethernet"
    - "optical interconnect AI datacenter"
    - "co-packaged optics"
    - "congestion control machine learning training"
    - "GPU communication scheduling"
    - "multi-rail GPU networking"
    - "GPU interconnect performance characterization"
    - "tensor parallelism communication overhead"
    - "gradient compression distributed training"
  last_run: 2026-08-08
---


## Summary

Large-scale LLM training and inference now depend on a hierarchy of interconnects: NVLink and NVSwitch inside servers, InfiniBand or RoCE Ethernet between them, and collective-communication libraries that turn raw fabric into training progress. The 2020–2026 evidence shows the lossless RDMA fabric that carried early GPT-era training under strain — congestion-control pathologies and head-of-line blocking are documented on both InfiniBand and RoCE — with hyperscale operators engineering around the problems and a lossy, packet-spraying Ultra Ethernet path emerging as the industry's answer. In parallel, Mixture-of-Experts models and long-context inference have made all-to-all traffic and KV-cache movement the new communication bottlenecks, and NVLink scale-up domains, UALink, and CXL memory pooling are reshaping where computation and memory physically live. Confidence is moderate: the transport and collective-communication stories are replicated across many independent studies, but the newest hardware (NVLink 4/5, UALink, GB200-class superpods) is covered mainly by vendor-adjacent or preprint evidence, and several load-bearing 2025–2026 results are single papers.

## Why this question

Interconnects are where large-scale LLM systems spend their second-largest budget after compute, and they set the scaling ceiling: a cluster's effective throughput is the product of GPU compute and the fraction of time communication leaves that compute idle. For training, that fraction is visible in every step-time breakdown — all-reduce alone can consume most of a step on commodity fabrics [120] — and for inference it has grown with context length, where a single million-token request can need a half-terabyte of KV cache moved across the cluster [106]. The question matters because the industry is currently placing three mutually incompatible bets: NVIDIA's NVLink scale-up domains, an open UALink/UEC counter-move, and incremental Ethernet scale-out. Which bet wins determines what operators buy, what model architectures are practical, and ultimately what tokens-per-watt the next generation of LLM infrastructure delivers.

The evidence base is young and fast-moving. Most of the corpus is from 2023–2026, much of it preprints, and it is split across three communities that rarely read each other — datacenter networking, ML systems, and computer architecture. A review that takes the field's own claims at face value would conclude everything is getting faster; a review that reads the measurements carefully finds a more interesting picture: the transport layer is being renegotiated, the collective layer is where most of the real gains are, and the newest hardware is being described almost entirely by the vendors who sell it.

## Scope and methods

**Question.** What does the evidence from 2020–2026 show about how interconnect technologies and their software stacks shape the performance, scaling, and cost of large-scale LLM training and inference?

**Inclusion criteria.** English-language papers from 2020-01-01 to 2026-12-31, peer-reviewed or arXiv preprints, whose central object is (a) an interconnect technology used in ML systems (InfiniBand, RDMA/RoCE, NVLink/NVSwitch, PCIe, CXL, UALink, Ultra Ethernet, optical), (b) the collective-communication or transport software on top (NCCL, MSCCL, TACCL, etc.), (c) topology-aware placement/scheduling for training clusters, or (d) communication-centric mechanisms in LLM training/inference (all-to-all for MoE, sequence/context parallelism, KV-cache transfer, prefill/decode disaggregation, gradient compression, in-network computing).

**Exclusion criteria.** Pre-2020 foundational work (DCQCN, InfiniBand's original design, Horovod) is discussed as context but never cited; vendor whitepapers and blog posts are excluded (vendor-authored conference papers are included but flagged); RDMA for databases, storage, and key-value stores is excluded unless the work targets LLM serving; wireless, quantum, and pure-device-level optical engineering without ML relevance is excluded.

**Search and screening.** Sixty-six literal query strings (listed in the frontmatter) were run against OpenAlex, Crossref, and the arXiv API on 2026-08-08 (Semantic Scholar was unavailable from this IP all session, as expected). The searches returned 3,635 unique works after deduplication. Screen 1 (title/topic scoring against an interconnect vocabulary) kept 1,868; manual curation against the criteria selected 144. Screen 2 (retrievability and DOI checks) dropped 7: five paywalled works with no open copy retrievable in-session (SmartCCL, HyGIN, NIKA, one MoE scheduling paper, one gradient-compression paper), and two USENIX papers with no resolvable DOI (OptiReduce, FLB). The final set is 137 sources: 60 read in full text, 77 at abstract level. Every DOI was verified against Crossref or resolved as `10.48550/arxiv.<id>` with a live resolution check; a Crossref retraction check (`update-to`) was run on all 95 publisher DOIs with zero retractions found.

**What this review deliberately does not cover.** The compute side of parallelism (model-parallel algorithm design per se), storage interconnects, and the electrical design of SerDes/PHY layers. It also does not cover the pre-2020 foundations of RDMA or NVLink except as attributed context.

## The landscape

The corpus is shaped by one fact: the field's center of gravity moved from HPC to ML systems between 2020 and 2024. The 2020 sources are mostly evaluations — InfiniBand EDR-vs-HDR on real workloads [137], PCIe-vs-NVLink-vs-NVSwitch on real GPUs [25], fabric benchmarks for data-distributed training [59], and the first LLM-scale parallelism systems [136]. By 2023–2024 the literature is dominated by systems papers that assume LLM workloads: collective-algorithm synthesis [133], topology-aware scheduling [77], MoE communication [65], and disaggregated inference [103]. By 2025–2026 the newest tier is explicitly about LLM scale: production Ethernet fabrics at Meta [1][132], 10,000-GPU training reports [85], and next-generation interconnect architectures [34][36].

Three communities contribute with different methods and standards of evidence. The datacenter-networking community contributes simulations and switch testbeds, often with impressive formal care but modest scale. The ML-systems community contributes end-to-end systems with measured speedups, but frequently evaluates on a single cluster configuration. The architecture community contributes chip-level designs (NVLink-C2C [23], optical interconnects [91]) whose performance claims are rarely independently verified. Hyperscaler field studies — the strongest evidence class — are rare: Meta's RoCE fabric [1] and 100 MW cluster [132], Google's TPU v4 fabric [87], and Alibaba's cluster work [118] are the anchors, and the review leans on them where possible.

The shape of the literature is also visibly vendor-influenced. NVIDIA-adjacent sources (including NVIDIA-authored papers, Hot Chips/ISSCC disclosures, and papers studying NVIDIA hardware) dominate the scale-up half of the corpus [24][23][45]; independent measurements of the same hardware are rarer and sometimes contradict the vendor picture [26][29]. This asymmetry is itself a finding: the newest interconnects are described by their sellers.

## Theme one — The transport layer: lossless RDMA fabric under pressure

The backbone of large-scale training is remote direct memory access (RDMA) running over a lossless fabric — InfiniBand in the HPC tradition, RoCE (RDMA over Converged Ethernet) in the Ethernet world. NCCL and its peers use the verbs API to move gradients and activations directly between GPU memories [38]. The evidence for this architecture's dominance is indirect but consistent: every large training system in the corpus assumes it [85][86][132], and the xCCL survey documents that every major vendor's collective library targets RDMA-capable fabrics [39].

The strain is documented from both sides of the fabric. On Ethernet, losslessness is implemented with Priority Flow Control (PFC), which stops the sender when buffers fill — and PFC-induced head-of-line blocking and deadlock-like pathologies are the recurring theme of the lossless-network literature. The strongest statement comes from Torsten Hoefler's group: "Datacenter Ethernet and RDMA: Issues at Hyperscale" argues that RoCE's design points — lossless, in-order delivery, ECN-based congestion control — are fundamentally mismatched with hyperscale operation, and that the industry should revisit them rather than patch them [3]. A companion position paper argues that RDMA congestion control only works when every tenant is "compliant," a fragile assumption in shared clouds [11]. On InfiniBand, the same underlying problems appear: head-of-line blocking in fat trees [20], congestion that monitoring must infer [19], and a documented latency-versus-bandwidth tradeoff at the switch [21].

The congestion-control literature is enormous and largely agrees on the diagnosis: ECN/RTT-based control reacts too slowly or too coarsely for incast-heavy AI traffic. The 2020–2026 wave proposes receiver-driven control [12], per-flow state [18], faster rate adaptation [14][5], and better buffer headroom [13], with adaptive routing as an InfiniBand-specific complement [22]. Notably, one of the few studies that measures congestion control's *end-to-end* effect on training finds that RoCE CC policy choice has little impact on DNN training throughput on their testbed [4] — a reminder that the CC literature's performance claims are mostly switch-level, not training-level.

The sharpest divide in the transport story is lossless versus lossy. A cluster of 2025–2026 work argues that RDMA should stop requiring losslessness: software-defined reliability with erasure coding and selective retransmission [7], forward error correction for lossy paths [10], fault-tolerant transport that absorbs link failures without stalling collectives [8], semantic-aware loss recovery at hyperscale QP counts [9], and a converged RDMA dataplane that mixes reliable and unreliable semantics [15]. Multi-path and multicast extensions fill out the picture [17][16]. Production reality, however, is still lossless: Meta's training fabric runs RoCE over a lossless Ethernet core and reports that the engineering investment is in making losslessness *fast and predictable* rather than abandoning it [1], and a 2025 workshop report describes the same approach for LLM training over RoCE [2].

The resolution of this tension is Ultra Ethernet, treated in Theme two below: a transport designed for lossy, packet-sprayed Ethernet with receiver-based reliability [34].

## Theme two — Scale-up and next-generation fabrics: NVLink domains, UALink, Ultra Ethernet

The most consequential architectural shift of the period is the scale-up interconnect. NVLink grew from a server-internal bus into a rack-scale fabric: the NVLink-C2C off-package interconnect provides coherent chip-to-chip links at 40 Gbps per pin [23], and NVSwitch-based "superpod" topologies turn hundreds of GPUs into a single high-bandwidth domain [24]. The TPDS evaluation of the earlier generation established the baseline — NVLink and NVSwitch beat PCIe by an order of magnitude for the all-reduce patterns that tensor parallelism needs [25] — and GEMM-level studies confirm the bandwidth gap persists across generations [29]. Independent work finds headroom even in NVLink: FlexLink reports 27% all-reduce and all-gather bandwidth gains by re-architecting how NVLink transfers are scheduled [26].

The scale-up domain changes the software stack in ways the corpus documents. Tensor-parallel communication, which previously had to cross the network, now stays inside the NVLink domain — the design premise of Megatron-LM's two-all-reduce-per-layer pattern [136] and of multi-node inference systems that keep TP intra-node and shard only what must cross [33]. NVLink-C2C is even proposed as a memory-disaggregation fabric in its own right, benchmarked against PCIe and network paths [27], and as the substrate for serverless LLM serving where the coherent link enables fast state sharing [30]. The newest generation is visible through two lenses: the GB200 NVL72 — a 72-GPU NVLink domain — where weight prefetch over NVLink replaces collective synchronization in MoE inference [131], and Meta's 100 MW cluster where 83,000 GB200s are networked with two 400 Gbps CX7 NICs per Grace CPU, doubling scale-out RDMA bandwidth per GPU relative to their reference design [132].

The counter-move is the open UALink specification, an industry consortium alternative to NVLink for scale-up domains, and Ultra Ethernet (UEC) for scale-out. UALink is described architecturally — 200 Gbps-per-lane links, low-latency load/store semantics — and compared head-to-head with NVLink by independent studies that find the gap closing on paper [28][36]. Ultra Ethernet's design is the more complete story: the consortium's position paper specifies packet spraying, a receiver-initiated transport, and lossy-fabric semantics to replace the lossless model of Theme one [34], with the consortium's own work framing it as the AI workload enabler [35]. The earliest evaluations are simulations — e.g., UEC transport's effect on co-tenant latency in reconfigurable NICs [37] — and no independent production-scale UEC deployment was in the literature as of the search date. The strategic stakes are clear from the corpus: scale-out Ethernet at Meta [1], scale-up NVLink at NVIDIA [45], and a consortium middle path [34] are three different answers to the same question, and the evidence cannot yet adjudicate between them.

## Theme three — The collective communication stack: where the real gains live

If the transport layer is contested, the collective layer is where the measured wins concentrate. NCCL — the de facto standard — remained opaque for years; the first in-depth public analysis of its protocols (Simple, LL, LL128), intra-node versus inter-node channels, and algorithm selection appeared in 2025 and confirmed that its behavior is a function of topology, message size, and hardware generation in ways that default configurations frequently get wrong [38]. The xCCL survey situates NCCL among industry libraries (MSCCL, RCCL, oneCCL, Gloo) and documents the fragmentation and tuning burden [39]. Microsoft's MSCCLang took the next step: a domain-specific language for expressing collective algorithms as communication sketches, compiled down to MSCCL implementations [40].

The synthesis line of work — automatically generating topology-aware collectives — is the strongest result class in this theme. TACCL synthesizes collective algorithms from a description of the physical topology (PCIe trees, NVLink meshes, InfiniBand rails) and beats NCCL by up to 6.7× on all-gather and all-to-all patterns, with end-to-end training speedups of 11%–2.3× [133]. Blink, the earlier and complementary line, packs spanning trees over heterogeneous NVLink/PCIe/NIC topologies and reports up to 8× faster model synchronization than the NCCL of its era [125]. The idea generalizes: TCCL co-optimizes collectives with traffic routing for GPU-centric clusters [44], and a reformulation of collective communication as a multi-commodity flow problem provides the theoretical framing [63].

Scheduling and overlap are the second result class. Canvas schedules collective communication at cluster scale [41]; Crux overlaps communication with compute at the kernel level [42]; Lagom hides all-reduce behind computation for distributed LLM training [55]; DeAR pipelines fine-grained all-reduce [52]. The 2026 generation pushes toward the physical limits: "Every Microsecond Matters" reports near speed-of-light all-reduce latency by minimizing protocol overheads [49], and "The Big Send-off" scales collectives by rethinking the endpoint model [50]. Compression keeps appearing as a bandwidth multiplier: gZCCL fuses compression into collective communication [43], ZeRO++ partitions parameters to cut collective traffic in zero-optimized training [53], FCDP caches parameters locally to avoid communication entirely [54], and even the all-reduce shape itself is being renegotiated [51]. At the system level, PCCL makes collective choice energy-aware [56] and SDCC makes it software-defined [57].

The scheduling line of work completes the picture: where collectives run is decided before training starts. The scheduling survey documents the field's shape [80]; ASTRA-sim2.0 provides the simulation infrastructure for exploring topology and scheduling jointly [84]; ring-all-reduce job scheduling with explicit communication-contention modeling shows the cluster-level cost of ignoring the fabric [81]; SortingHat places models on multi-GPU systems using the actual system topology [78]; SpiderFlow extends topology-aware scheduling to decentralized LLM training across clusters [79]; and logical/physical topology-aware collectives show that matching collective shape to the physical layout — not the logical ring — is where the wins are [83].

Reliability and observability became first-class concerns as clusters grew. Two independent 2025–2026 lines address the same failure mode — one slow or failed link stalling a synchronized collective: "Don't Let a Few Network Failures Slow the Entire AllReduce" redesigns the failure path [60], and a production-oriented library from a major operator reports reliability and observability as design goals [61], alongside deadlock-prevention analysis [62] and the NCCLbpf policy layer [47]. Observability tooling arrived with NIXT, an NCCL inspector/exporter [48]. The abstraction layer is also being re-architected: GPU-initiated networking removes the CPU from the communication path [64], NCCL EP unifies expert-parallel communication under NCCL [46], and UBEP — accepted to SIGCOMM 2026 — re-architects MoE all-to-all for production superpods, cutting all-to-all latency by up to 52.4% [45].

The evidence also includes the negative results that keep the theme honest: communication-characteristic studies of real distributed training show that the traffic mix is dominated by a few collective shapes and that overlap quality, not raw bandwidth, explains most observed variance [58][119].

## Theme four — Mixture-of-Experts: the all-to-all bottleneck model architecture created

The single most consequential workload shift for interconnects is MoE. Dense models communicate with all-reduce; MoE models additionally route tokens to experts across the cluster with all-to-all exchanges, and the literature agrees this is a first-class bottleneck. Tutel's adaptive parallelism [65] and DeepSpeed-MoE's training/inference stack [66] established the problem; DeepSeek's production reports made the magnitude concrete — DeepSeek-V2's architecture was explicitly co-designed with expert-parallel all-to-all communication and communication-balance losses [67], DeepSeekMoE pushes expert specialization with auxiliary communication control [68], and DeepSeek-V3's training framework reports "near-zero all-to-all communication overhead" through DualPipe computation-communication overlap and customized cross-node all-to-all kernels [69].

The optimization lines are consistent across independent groups. Overlap: HALO hides all-to-all behind expert computation via locality-aware scheduling [71]. Communication reduction: LSH-MoE routes by locality-sensitive hashing so experts can be placed near their tokens [70], and Shortcut-connected expert parallelism cuts activation traffic by skipping the sequential dependency between all-to-all rounds [76]. Load balancing: FEPLB exploits GPU copy engines to make MoE load balancing nearly free [72]. Placement: cross-pod communication overhead is attacked with hybrid parallelism in multi-tenant clusters [73]. And two radical options: dynamic in-switch computing inside the network dataplane to aggregate MoE traffic [74], and disaggregated expert parallelism in serving, where experts live on separate machines and are fetched on demand [75].

The disagreement underneath (see also "Where the evidence disagrees") is whether all-to-all is fundamentally expensive or just under-engineered: DeepSeek's production experience says the latter at their scale [69], while the optimization literature — and the superpod re-architecture [45] — treats it as the central cost.

## Theme five — Long context: sequence and context parallelism over the network

Long-context training and inference changed the communication pattern again. Tensor parallelism's all-reduce traffic scales with sequence length, which is why sequence-parallel variants split the sequence dimension instead [112] — a design that DeepSpeed-Ulysses and Megatron's context parallelism made practical, and that the systems in this review benchmark against [114][115]. Ring attention added the key theoretical result: KV-block communication can be fully overlapped with attention compute, so long-context scaling is governed by the bandwidth-to-FLOPS ratio, not raw bandwidth [134]. Striped Attention improves ring load-balancing for causal transformers, reporting up to 1.45× end-to-end throughput over ring at 256K sequence length [113].

The 2024–2026 generation pushes the same ideas further with explicit interconnect engineering. LoongTrain's 2D attention uses head-parallelism and context-parallelism together so that both NICs and NVLink are saturated, reporting up to 2.88× MFU over Ulysses-style baselines [114]. Untied Ulysses (UPipe) cuts the all-to-all activation memory by up to 87.5% with headwise chunking, reaching 5M-token context on a single 8-GPU node [115]. DCP makes context partitioning dynamic to match variable input lengths [116]. On the inference side, context parallelism prefill at 128 GPUs achieves 1M-token prefill of Llama-3-405B in 77 seconds with near-linear scaling over both RDMA and TCP interconnects [135]. The older pipeline-parallel line contributes the complementary lesson: TeraPipe's token-level pipelining cut communication stalls in model-parallel training [111], and Megatron's interleaved scheduling pushed composition of parallelisms to thousands of GPUs [117]. Whale's hardware-aware placement — including NVLink-aware all-reduce placement — showed 95% scaling efficiency on 512 GPUs for a 10T-parameter model [118].

The through-line: as context grows, the network becomes a KV-movement problem as much as a gradient-movement problem, and the systems that win are the ones that overlap that movement with compute [134][106].

## Theme six — Fabric architecture: optical circuit switching, memory pooling, and CXL

Beyond electrical packet switching, two architectural alternatives recur: optical circuit switching (OCS) and memory-centric interconnects.

**Optical.** The evidence that OCS works at production scale is Google's TPU v4 — an optically reconfigurable supercomputer that uses OCS to reshape the fabric between training jobs, improving utilization and resilience [87]. Microsoft's Apollo program is the second production datapoint, landing OCS at datacenter scale with circulators to double switch radix [126]. The research wave generalizes the idea for ML: Lightwave Fabrics builds OCS-based fabrics specifically for machine-learning workloads [88]; InfiniteHBD proposes datacenter-scale high-bandwidth domains with OCS for LLM training [89]; Mercury combines optical timeslot switching with circuit switching [90]; and a 2026 study co-optimizes OCS reconfiguration schedules with LLM training phases [82]. On the device side, multi-channel optical interconnects for multi-GPU systems [91] and co-packaged optics roadmaps [92] describe the path to higher bandwidth density, while the 2020 overview of interconnection networks for DNN accelerators remains a useful taxonomy [93]. The honest summary from the corpus: one production success (TPU v4 [87]), strong testbeds, and no independent large-scale LLM deployment yet.

**Memory pooling and CXL.** The memory-side argument is that LLM capacity needs exceed per-GPU HBM, and interconnects are the solution. CXL's promise — coherent memory semantics over a PCIe-class physical layer, with lower latency than network RDMA [128] — is documented with real hardware by the first CXL-ready systems study, which shows true CXL devices differing from emulation and requiring allocation-policy changes for bandwidth-hungry workloads [127]. Pooling is the production-ish story: Pond describes CXL-based memory pooling for cloud platforms [101], memory pooling with CXL is surveyed as an architecture [95], hyperscale adoption is reported with compression to stretch capacity [129], and cache-coherent disaggregated memory has been evaluated for deep learning training [100]. The LLM-specific evidence is more speculative: a GPU-free CXL-memory inference system [96], CXL memory expansion for retrieval-augmented generation [98], bandwidth-efficient compression for CXL expansion [97], and dynamic KV-cache placement across heterogeneous memory tiers [99] are all emulation- or simulation-based. The newest work connects the two halves of this theme: topology-aware data movement for disaggregated GPU inference treats memory and network placement as one problem [102].

The energy argument ties the fabric to the bottom line: a 2025 analysis argues that communication tax — not compute — is what now prioritizes memory and interconnect design in modern AI systems [94].

## Theme seven — Disaggregated inference: the prefill/decode split and KV-cache movement

Inference disaggregation is the clearest example of an interconnect-motivated architecture change. Prefill and decode have different compute and memory profiles, so serving systems split them onto different machines — and pay for it with the need to move KV cache between them. DistServe's OSDI paper established the goodput argument, reporting 2.1× higher goodput than monolithic serving under SLO constraints [103]; Splitwise made the phase-splitting case for cost efficiency [104]. Mooncake's KVCache-centric architecture — treating the KV cache as a disaggregated, network-accessible tier — became the reference design for 2025-era serving, with the ACM ToS version documenting the transfer economics over RDMA [105]. LMCache generalizes the idea as a caching layer spanning GPU/CPU/storage/network tiers, reporting up to 15× throughput over baseline serving [108]. ServerlessPD pushes the transport further, co-designing RDMA with disaggregated prefill-decode so that KV state transfers with near-zero-copy [107]. SYMPHONY moves KV migration off the critical path with advisory scheduling [130].

The dissenting results are as important as the consensus. LoongServe shows that elastic sequence parallelism — keeping prefill and decode together but rebalancing the sequence dimension — beats prefill/decode disaggregation by up to 5.81× on long-context workloads, precisely because it avoids most KV migration [106]. A 2026 analysis goes further, arguing that the prefill/decode dichotomy itself is too coarse and that operator-level disaggregation across heterogeneous platforms is the next step [109]. At the small scale, communication-efficient model parallelism for in-situ (edge) inference shows that restructuring communication — not disaggregating — can win when interconnects are weak [110]. The synthesis: disaggregation pays when KV transfer is cheap relative to the compute imbalance it fixes; the corpus's numbers bracket that tradeoff [103][105][106].

## Theme eight — Reliability, security, and the cost of moving bytes

Four cross-cutting concerns complete the picture. **Reliability:** synchronized collectives are fragile — one slow link stalls all-reduce [60], deadlocks are a real hazard in collective scheduling [62], and RDMA's reliability model is being renegotiated for lossy fabrics [6][8]. **Security:** the interconnect is a new side-channel surface — NVBleed demonstrates a covert channel over NVLink contention at 70.59 Kbps with 4.78% error [31], and the earlier contention-based attack on multi-GPU interconnects showed the same class of vulnerability [32]. **Bandwidth scarcity:** gradient compression remains the blunt instrument for bandwidth-constrained clusters, with 1-bit LAMB's 4.6× communication reduction and 2.8× end-to-end speedup on Ethernet clusters [120] and near-lossless compression at up to 67.1% volume reduction [121] as the representative results. **Energy:** the communication tax argument [94] and energy-aware collective scheduling [56] are the only direct measurements of interconnect energy in the corpus — a striking gap given the topic's importance to datacenter power budgets.

In-network computing sits at the intersection: offloading aggregation into the network — host-driven in-network aggregation on RDMA [122], constrained in-network computing with congestion guarantees [123], and the survey of in-network computing for distributed ML [124] — promises to cut both latency and energy, at the price of programmable-switch complexity that production LLM clusters have not yet widely accepted.

## Where the evidence disagrees

**Lossless versus lossy RDMA.** The position literature says losslessness is the problem [3][11], the reliability-engineering literature builds the lossy future [7][10][15], and the production evidence — Meta's lossless RoCE fabric [1] — shows losslessness working at scale with heavy engineering. The apparent conflict dissolves once separated by role: the critics diagnose the *default* lossless stack; the operators show what *managed* losslessness achieves; the lossy work is younger and unproven at LLM scale. Ultra Ethernet is the industry's attempt to have both [34]. The remaining open question is whether UEC's lossy semantics actually deliver at 100,000-GPU scale — no evidence exists yet.

**Does congestion control matter for training?** The CC literature assumes it does [18][12][14], but the one training-level measurement found policy choice barely moves end-to-end training throughput [4]. Explanation: CC matters at the fabric's saturation point — large clusters with incast from synchronized all-reduce — and the measurement was small-scale. Both camps are consistent with "CC matters at scale"; neither has measured it at the scale where it matters.

**Scale-up versus scale-out.** NVIDIA's bet is that NVLink domains win [45][131]; Meta's field data shows Ethernet scale-out working for them [1][132]; UALink's backers argue the open path [28][36]. The evidence cannot resolve this because the load-bearing sources are vendor-adjacent, the comparisons are on paper [28], and the production datapoints are single operators. This is a live strategic disagreement, and the review treats it as such.

**Is MoE all-to-all a fundamental bottleneck?** DeepSeek's production reports claim near-zero all-to-all overhead with co-designed kernels and overlap [69], while the optimization literature treats all-to-all as the dominant cost [71][72][45]. The reconciliation is hardware- and software-generation-dependent: DeepSeek's result is on their fabric with bespoke engineering; the optimizations target the general case. The 52.4% latency cut reported by UBEP on production superpods [45] suggests the general case still has a long way to go.

**Disaggregate or not.** Prefill/decode disaggregation wins for mixed workloads [103][104][105], but long-context serving favors elastic sequence parallelism [106], and the newest work says the dichotomy itself is wrong [109]. Population explains it: the disagreement is about workload — short-context chat favors disaggregation; long-context favors SP; agentic mixes break both.

## Gaps and open questions

- **No independent measurements of the newest scale-up fabrics.** NVLink 4/5, UALink, and GB200 NVL72-class domains are described by vendors, vendor-adjacent papers, and one hyperscaler [132][131][28]. An independent, reproducible benchmark of NVLink-domain versus UALink versus Ethernet for the same training workloads would settle more than any other single experiment.
- **Lossy RDMA at LLM scale is untested.** The lossy-fabric results [7][8][9] are prototypes and simulations; no published production deployment exists. Ultra Ethernet's central claims [34] rest on the same gap.
- **Interconnect energy is nearly unmeasured.** Two sources [94][56] are the entire corpus on tokens-per-watt effects of networking; the datacenter energy literature treats compute, not fabric, and the review found no field study of interconnect energy share in an LLM cluster.
- **KV-transfer economics at extreme scale.** The KV-cache movement numbers come from three systems [105][108][106]; the paywalled NIKA analysis of optimal KV transfer could not be retrieved for this review, and no independent cost model exists.
- **CXL for LLM is emulation-only.** Every CXL-for-inference result [96][97][98][99] is simulation or emulation; production CXL memory pooling for LLM serving is unreported.
- **The collective stack lacks a standard benchmark.** The xCCL survey documents fragmentation [39]; the field's speedup claims (6.7× [133], 8× [125], 27% [26], 52.4% [45]) are not comparable across papers because no common topology, workload, and NCCL-version baseline exists.
- **Optical OCS beyond TPU v4.** The production evidence is a single operator's two deployments [87][126]; everything else [88][89][90] is testbed-scale.

## Confidence and limitations

**Confidence.** Moderate overall, with a clear gradient. High confidence: the transport-layer diagnoses (lossless-fabric pathologies, CC limitations) are replicated across many independent groups and both fabric families [3][20][4]; the collective-communication wins are measured end-to-end in multiple systems [133][41][55]; MoE all-to-all as a first-class bottleneck is agreed across vendors, hyperscalers, and academics [65][67][45]. Moderate-to-low confidence: everything about the 2025–2026 hardware generation — UALink, UEC, NVL72 — rests on position papers, simulations, or single-operator reports [28][37][36].

**Limitations of this review.** 77 of 137 sources were read at abstract level (many ACM/IEEE papers are paywalled); numbers from those are hedged and stated as reported. The search was English-only, 2020-2026, and excluded gray literature — vendor whitepapers contain fabric details (e.g., NVLink 5 specifications) that peer-reviewed sources do not, so the newest-hardware picture is thinner than the industry's own documentation. OpenAlex's daily budget was exhausted mid-session and the search was carried by Crossref and the arXiv API thereafter; five selected sources (including one KV-transfer analysis and one collective-scheduling paper) were dropped as unretrievable. Semantic Scholar contributed nothing (429-limited from this IP all session). Finally, the corpus is concentrated in the 2023–2026 window — the field is young, and "consensus" statements here are consensus about a fast-moving target.

## Evidence table

| key | design | sample | measure | finding | limitations | confidence | access | note |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| gangidi2024ethernet | systems paper | Meta's production RoCE (RDMA over Converged Ethernet) networks for distributed AI training | network reliability, scalability, operational efficiency | Presents the design, implementation, and operation of Meta's RoCE networks for distributed AI training; no quantitative results are reported in the abstract. | Abstract-only; operational details and numbers not available in the record. | moderate | abstract-only | Production evidence that RoCE/Ethernet is viable as the interconnect fabric for large-scale AI training. |
| batllecas2025scaling | benchmark | Production clusters running large-scale LLM training workloads; RoCE vs InfiniBand | scaling performance, throughput, training efficiency | RoCE achieves near-linear scaling performance comparable to InfiniBand when properly configured, per extensive experiments on production clusters (no specific numbers in abstract). | Abstract-only; no numeric speedups or cluster sizes reported. | moderate | abstract-only | Head-to-head benchmark of RoCE vs InfiniBand for LLM training, supporting cost-effective Ethernet-based interconnects. |
| hoefler2023datacent | survey | RoCE/RDMA design points across hyperscale datacenters; analysis uses 800 Gb/s links, 600 ns hop latency, 9 kB frames | headroom buffer requirements, packet rates, header overhead, retransmission bandwidth loss | RoCEv2 headers total 66 B vs 20 B for InfiniBand, cutting the max packet rate on an 800 Gb/s link from 12.5 to 1.4 Gpps for 8-byte messages, and the paper predicts modernized Ethernet will replace TCP and RoCE within a decade. | Position/vision paper by industry authors, not an empirical evaluation; quantitative examples are illustrative. | high | full-text | Foundational critique of RoCE/PFC (headroom buffers, victim flows, go-back-N, ECN) that motivates Ultra Ethernet for AI datacenters. |
| khan2022impact | simulation | ASTRA-SIM + NS3 simulation of DLRM training on a 2-level Clos platform: 8 GPUs/server over 200 GB/s NVLink, 200 Gbps NIC-to-TOR links, up to 16 racks | end-to-end training iteration time, exposed communication time, PFC PAUSE frames, switch buffer occupancy | State-of-the-art RoCE congestion control schemes (DCQCN, DCTCP, TIMELY, HPCC) have little impact on end-to-end training performance versus baseline PFC on distributed training platforms. | Simulation only (ASTRA-SIM/NS3); single workload (DLRM) and single-tenant assumption; no hardware testbed. | high | full-text | Shows general-datacenter congestion control does not transfer to collective-based (All-Reduce/All-To-All) training networks, guiding interconnect CC design. |
| he2025adaptive | systems paper | Testbed experiments on RDMA data center networks (lossless, PFC-based) | flow completion time (FCT), convergence speed, transmission rate | FACC, a switch-driven RDMA congestion control algorithm, improves convergence speed while achieving at most 86.6% lower flow completion time compared with state-of-the-art approaches. | Abstract-only; testbed scale and hardware not stated. | moderate | abstract-only | RDMA congestion control optimization for latency-sensitive AI applications on lossless fabrics. |
| li2025revisiti | systems paper | P4 switch prototype (DCP-Switch) and FPGA RNIC prototype (DCP-RNIC) on high-speed lossy fabrics | throughput and performance relative to SOTA lossless and lossy RDMA solutions | DCP, a PFC-independent transport co-designing switches and RNICs, achieves 1.6x and 2.1x performance improvements compared to state-of-the-art lossless and lossy RDMA solutions, respectively. | Prototype hardware (P4/FPGA), not production ASICs; abstract-only details. | moderate | abstract-only | Enables efficient RDMA over lossy (non-PFC) Ethernet fabrics, relevant to RoCE scalability for LLM training. |
| khalilov2025software | systems paper | SDR-RDMA on NVIDIA BlueField-3 SuperNIC (DPA offload) plus simulation; measured 350 km inter-DC link (100 Gbit/s, drop rates 1e-4 to 1e-1) and links up to 3.2 Tbit/s | average and 99.9th-percentile RDMA Write completion time, message completion time, packet drop rate | Guided selection between Selective Repeat and Erasure Coding reliability improves average and 99.9th-percentile RDMA Write completion time by up to 5x and 12x respectively on long-haul links, with the SDR backend sustaining line-rate packet processing up to 3.2 Tbit/s. | Requires BlueField-3 DPA offload; inter-datacenter (lossy, ms-RTT) focus rather than intra-cluster; partially simulation-based. | high | full-text | Software-defined reliability stack for planetary-scale RDMA, key for multi-datacenter LLM training collectives. |
| lin2025shift | systems paper | rdma-core userspace implementation evaluated with PyTorch distributed training; cites Alibaba (15.8% of training failures network-related), Azure (8.3% of InfiniBand failures), Tencent (15% NIC, 30% switch/fiber) production data | overhead during normal operation, failure masking capability, training continuity | SHIFT proves an RDMA Failover Trilemma (exactly-once execution, receiver-NIC opacity, and zero-copy cannot coexist) and, exploiting idempotent bulk transfers in NCCL/NVSHMEM, masks fatal NIC failures and link anomalies with negligible overhead, avoiding costly training restarts. | Does not support NCCL LL/LL128 or atomic-notification protocols; backup-NIC path is constrained by PCIe bandwidth and co-located traffic. | high | full-text | Extends RDMA fault tolerance to cross-NIC level and quantifies network failures' share in large-scale LLM training. |
| hua2026semantic | systems paper | Testbed with Mellanox ConnectX-5 RNICs (4 hosts, 100 Gbps switch) plus ns-3 simulations of large-scale collective communication | 99th-percentile flow completion time, QP scalability, RNIC throughput, PCIe bandwidth overhead | CSA-UD, a communication-semantic-aware unreliable-datagram loss recovery scheme, achieves over 30% lower 99th-percentile flow completion time than UD counterparts and better scalability than RC, whose throughput degrades once concurrent QPs exceed roughly 512 due to PCIe metadata-fetch penalties. | Software-based recovery runs on the host CPU; requires UD mode and per-flow bitmap state; RNIC cache limits (~5,000 QPs) motivate design. | high | full-text | Addresses QP scalability and tail latency of RDMA collectives (All-Reduce/All-to-All) for trillion-parameter training. |
| yang2025adaptive | systems paper | RDMA data center networks sensitive to packet loss (Go-Back-N retransmission); FEC-based scheme | packet loss recovery effectiveness, throughput (not reported) | Proposes Forward Error Correction with adaptive redundancy to recover lost packets and mitigate Go-Back-N retransmission costs; the abstract is truncated and reports no quantitative results. | Abstract truncated; no numbers, hardware, or evaluation details available. | low | abstract-only | FEC-based alternative to lossless fabrics for RDMA, relevant to RoCE reliability design. |
| snyder2023congesti | field study | RDMA deployed in shared cloud environments; congestion control enforcement and algorithms | short flow completion time, attacker bandwidth, congestion severity | Performance hacking attacks exploiting RDMA congestion control can give attackers extra bandwidth and increase short flow completion times by several orders of magnitude, exposing a fundamental tradeoff between short-flow latency and performance isolation. | Abstract-only; no experimental setup or numbers reported. | moderate | abstract-only | Security/isolation angle on RDMA congestion control for shared AI clouds. |
| wan2025revisiti | systems paper | Commodity servers and RNICs; RDMA intra-host network experiments | network throughput and latency under intra-host congestion | RHCC, an RDMA intra-host congestion control scheme with sub-RTT-granularity avoidance and proactive RNIC traffic adjustment, can increase throughput and decrease latency by up to 2x and 1.4x, respectively. | Abstract-only; intra-host scope; inter-host CC interaction only briefly covered. | moderate | abstract-only | Highlights intra-host congestion as an emerging bottleneck as RNIC speeds outpace host interconnects. |
| shan2026headroom | systems paper | PFC-enabled commodity Ethernet switches; dynamic/shared headroom allocation (DSH) | flow completion time, buffer utilization, PFC message frequency | DSH, which dynamically allocates and shares per-queue headroom under port-level flow control, reduces flow completion time by up to ~78.8% in extensive experiments. | Abstract-only; switch model and scale not stated. | moderate | abstract-only | Headroom buffer inefficiency in lossless Ethernet, a core cost of PFC-based RoCE deployments. |
| liu2026accurate | systems paper | DPDK prototype of FAR rate-control protocol on datacenter networks | tail flow completion time (FCT), bandwidth estimation accuracy, queue oscillation | FAR, which couples bandwidth-estimation-based congestion control with end-to-end flow pausing, reduces tail FCT by up to 67% compared with state-of-the-art designs. | DPDK software prototype; abstract-only details. | moderate | abstract-only | Combines flow pausing and congestion control for low-latency datacenter (incl. RDMA) networks. |
| planeta2025converge | systems paper | RDMA networks in cloud/HPC settings; CoRD removes kernel bypass from the RDMA dataplane | end-to-end application performance, OS-level control overhead | CoRD shows kernel bypass is not mandatory for high-performance networking and can improve end-to-end application performance by up to 10% in one use case while enabling OS control of the RDMA dataplane. | Abstract-only; single demonstrated use case. | moderate | abstract-only | Challenges the kernel-bypass assumption of RDMA stacks, relevant to cloud-hosted RDMA AI training. |
| li2023gleam | systems paper | Fully functional Gleam prototype: switch with extended multicast forwarding table + 4 commodity servers with unmodified RNICs; large-scale simulations | multicast communication time, data replication throughput, HPL communication time | Gleam, which re-purposes RDMA RC logic with switch coordination for in-fabric multicast, achieves 2.9x lower communication time for the HPL benchmark and 2.7x higher data replication throughput (up to 2.2x multicast speedup on testbed). | Requires switch modifications (layer-4 states in forwarding table); evaluated on HPC/storage multicast patterns rather than LLM workloads. | high | full-text | Brings reliable multicast to RDMA RC transport, a building block for group communication in training frameworks. |
| tian2021multi | systems paper | Maestro software middleware deployed on existing RDMA NICs for distributed deep learning (DDL) applications | multi-path utilization, CPU overhead, DDL application performance | Maestro, a software-defined multi-path RDMA solution, fully utilizes multiple paths with negligible CPU overhead and enhances DDL application performance (no numeric results in abstract). | Abstract-only; no numbers; user-space middleware overhead not quantified in record. | moderate | abstract-only | Software multi-path load balancing for RDMA, addressing the single-path limitation of RNIC hardware. |
| luo2024flowstar | systems paper | InfiniBand networks (per-flow CBFC on switches); evaluation across different workloads | average and tail message completion time | FlowStar, a per-flow accurate InfiniBand congestion control combining optimized per-flow CBFC with rate adjustment, improves average and tail message completion time under different workloads (no numeric values reported). | Abstract-only; no numbers; requires per-flow state on switches. | moderate | abstract-only | InfiniBand congestion control addressing victim flows and slow-convergence congestion spreading. |
| cascajo2023monitori | field study | Real InfiniBand-based cluster combining LIMITLESS monitoring with OpenSM control software | number of wrong reactions of the InfiniBand congestion-control mechanism | Dynamically reconfiguring InfiniBand congestion-control parameters based on monitoring data significantly reduces the number of wrong reactions to the congestion-control mechanism (no numbers reported). | Abstract-only; no quantitative results; parameter tuning approach is cluster-specific. | moderate | abstract-only | Operational tuning of InfiniBand congestion control parameters in real HPC clusters. |
| maglionem2020path2sl | systems paper | InfiniBand fat-tree network topologies; Path2SL Virtual Lane queuing scheme | head-of-line (HoL) blocking reduction, network performance under congestion | Path2SL optimizes the use of InfiniBand Virtual Lanes to reduce HoL blocking in fat-tree topologies (no numeric results in abstract). | Abstract-only; no numbers; evaluation context (simulation vs testbed) not stated. | moderate | abstract-only | InfiniBand queuing/Virtual Lane mechanisms for congestion mitigation in HPC interconnects. |
| katebzade2020evaluati | benchmark | Rack-scale InfiniBand cluster benchmarked with RPerf in isolated and mixed latency/bandwidth traffic scenarios | switch latency, bandwidth, and latency-bandwidth tradeoff under mixed traffic | The evaluated InfiniBand switch can provide either low latency or high bandwidth, but not both simultaneously in mixed-traffic scenarios, and none of the evaluated improvement options is ideal. | Single switch model; measurement tool (RPerf) without hardware support; abstract-only. | moderate | abstract-only | Measurement methodology (RPerf) and evidence on InfiniBand switch behavior under mixed workloads. |
| luo2024alarm | simulation | Simulated HPC networks with InfiniBand-style deterministic routing; one-way-delay-guided adaptive routing | average and p99 flow completion time (FCT) slowdown | Alarm, a probabilistic adaptive routing system using one-way delay metrics with per-flowlet decisions, achieves 2.0x and 1.7x better average and p99 FCT slowdown than existing adaptive routing. | Simulation-based evaluation; abstract-only details. | moderate | abstract-only | Adaptive routing to balance traffic in InfiniBand-dominated HPC/AI clusters. |
| wei2023nvlink | survey | NVIDIA Grace-Hopper and Grace Superchip systems using NVLink-C2C | link bandwidth, cache-coherent memory access | NVLink-C2C provides a 900 GB/s link between Grace and Hopper (or two Grace chips) with a unified, cache-coherent memory address space combining system and HBM GPU memories. | Vendor product overview; abstract-only; no independent measurements. | moderate | abstract-only | Documents NVLink-C2C as the high-bandwidth CPU-GPU interconnect enabling unified memory for AI/HPC. |
| ishii2022nvlink | survey | Historical overview of NVLink generations | not reported | The abstract contains only the title 'Brief History of NVLink'; no content or numbers are available in the record. | Abstract effectively empty; cannot extract findings. | low | abstract-only | Historical context source for NVLink; minimal extractable evidence. |
| li2020evaluati | benchmark | Six platforms: P100-DGX-1, V100-DGX-1, DGX-2, OLCF SummitDev and Summit, SLI system with 2x RTX-2080; interconnects PCIe, NVLink-V1/V2, NVLink-SLI, NVSwitch | GPU communication bandwidth, NUMA effects, multi-GPU application performance | The evaluation identifies four new types of GPU communication network NUMA effects (three triggered by NVLink topology/connectivity/routing and one by a PCIe chipset design issue), showing GPU choice can considerably impact communication efficiency (no numeric bandwidths in abstract). | Older hardware generations (P100/V100/RTX-2080 era); abstract-only. | moderate | abstract-only | Empirical characterization of intra-node GPU interconnects (PCIe/NVLink/NVSwitch) foundational to multi-GPU training. |
| shen2025flexlink | systems paper | 8-GPU H800 server (NVLink 400 GB/s, PCIe Gen5 x16 ~128 GB/s, 800 Gb/s NICs); MoE training and long-sequence inference workloads | AllReduce/AllGather bandwidth, communication time share, load distribution | FlexLink aggregates NVLink, PCIe, and RDMA NICs with two-stage adaptive load balancing, improving AllReduce and AllGather bandwidth by up to 26% and 27% over NCCL by offloading 2-22% of traffic to previously idle links; MoE communication can consume up to 43.6% of forward-pass time and Flash Communication overhead up to 65.9%. | H800-specific (reduced NVLink); PCIe path contention caps gains; gains depend on idle secondary-link bandwidth (14-33% across surveyed platforms). | high | full-text | Directly addresses intra-node interconnect utilization (NVLink+PCIe+RDMA aggregation) for LLM training/inference collectives. |
| werner2025memory | benchmark | NVIDIA Grace-Hopper system; CPU accessing GPU memory via NVLink-C2C for memory expansion | memory access throughput and latency; database index operation performance | CPU access to GPU memory over NVLink-C2C achieves throughput of up to 168 GB/s with access latencies between about 800 ns and 1000 ns on the Grace-Hopper system. | Single system generation; microbenchmark scope; abstract-only. | moderate | abstract-only | Quantifies NVLink-C2C viability for memory expansion, informing memory-disaggregation interconnect choices. |
| vemuri2025centric | theoretical | Analytical models of latency- and bandwidth-sensitive transformer collectives over UALink 1.0 (up to 1024 accelerators/pod, <1 us request-to-response) vs NVLink 3-5 (NVLink 5 ~1.8 TB/s per-GPU) | step-time contribution of collectives under flat and hierarchical topologies | A flat UALink pod can reduce the step-time contribution of small-message collectives at scales up to 1024 GPUs, while NVLink 5 maintains an advantage in bandwidth-dominated regimes due to aggregate per-GPU bandwidth near 1.8 TB/s and large non-blocking pods. | Analytical/figure-generation models, not empirical measurements; abstract-only. | moderate | abstract-only | Compares open UALink vs proprietary NVLink for AI-training collectives, informing interconnect standardization. |
| choi2022multi | benchmark | Multi-GPU servers with up to 8 A100 GPUs connected by NVLink 3.0 links through NVSwitch | matrix-matrix multiplication performance | Analyzes the performance of matrix-matrix multiplication on NVLink 3.0/NVSwitch-connected 8-A100 servers; the abstract is truncated and reports no numeric results. | Abstract truncated; no numbers available. | low | abstract-only | Intra-node NVLink/NVSwitch performance analysis for multi-GPU compute kernels. |
| luo2026c2cserve | systems paper | GH200 Superchips with MIG partitioning; three-week Alibaba production trace of 89 models; Llama 3 series workloads | cold-start latency, TTFT/TPOT attainment, throughput under shared NVLink-C2C contention | C2CServe keeps model weights in CPU memory streamed over NVLink-C2C (~450 GB/s per direction, ~7x PCIe 5.0 x16) to MIG instances, reducing cold-start latency by up to 7.1x for dense and 4.6x for MoE models versus state-of-the-art serverless systems while maintaining over 95% TTFT/TPOT attainment under C2C contention. | GH200-specific; C2C bandwidth is shared across MIG instances (interference gap widens 28% to 42% as co-located model sizes grow 5 GB to 44 GB); requires Superchip hardware. | high | full-text | Shows NVLink-C2C bandwidth reshaping LLM inference/serving architecture, an interconnect-centric serving system. |
| zhang2025nvbleed | case study | DGX-1 (8x Tesla P100, NVLink-V1) and GCP 8x Tesla V100 (NVLink-V2) cloud instances; 18 HPC/DL applications; 50 Blender characters | covert channel bandwidth and error rate; side-channel F1/accuracy scores | NVBleed exploits NVLink contention timing and performance counters to build covert channels achieving 70.59 Kbps with 4.78% error rate, application fingerprinting with F1 up to 97.78%, Blender character identification with 91.56%, and cross-VM attacks on GCP exceeding 88% F1. | Tested on NVLink-V1/V2 only (not V3/NVSwitch); performance-counter leakage can be restricted by a NVIDIA driver patch; contention-based channel needs co-location. | high | full-text | Security analysis of the NVLink interconnect in multi-GPU and cloud AI systems, documenting leakage via contention and counters. |
| zhang2024beyond | case study | Multi-GPU systems sharing the NVLink interconnect (CPU-GPU and GPU-GPU) | covert channel bandwidth and error rate; application fingerprinting accuracy | A covert channel attack across two GPUs achieves 45.5 kbps with a low error rate, and NVLink congestion monitoring enables application fingerprinting without special permissions. | Abstract-only; fewer experimental details than the companion NVBleed study. | moderate | abstract-only | Demonstrates NVLink congestion leakage as a side channel in multi-GPU systems. |
| singhania2025understa | benchmark | Perlmutter (A100, intra-node NVLink, inter-node HPE Slingshot-11) and Vista (GH200, InfiniBand); Llama 3.1 70B/405B with vLLM, SGLang, and YALIS engines; 4 to 128 GPUs | end-to-end batch latency, all-reduce latency, strong-scaling efficiency | NVRAR, a hierarchical NVSHMEM-based recursive-doubling all-reduce, achieves up to 1.9x-3.6x lower latency than NCCL for 128 KB-2 MB messages on Slingshot/InfiniBand and up to 1.72x lower end-to-end batch latency for Llama 3.1 405B tensor-parallel decode-heavy inference, while NCCL inter-node all-reduce is 1.5-2x slower than MPI for 512 KB-1 MB messages. | Preliminary work (under review); decode-heavy focus; YALIS is a research prototype; scaling experiments limited to two HPC systems. | high | full-text | Shows all-reduce/inter-node interconnect (Slingshot, InfiniBand) as the key bottleneck in multi-node LLM inference. |
| hoefler2025ultra | survey | Ultra Ethernet (UE) 1.0 specification (562 pages) authored by UEC members; backend networks at 400+ Gbps over 10-150 m links | design parameters: connectionless transport (UET), ECMP packet spraying via entropy values, PDC reliability, NSCC/RCCC congestion control, TSS encryption, CBFC/LLR link extensions | UE 1.0 replaces connection-oriented, lossless (PFC-based) RDMA/RoCE semantics with a connectionless, packet-sprayed transport designed for millions of endpoints, with ECMP hash-conflict probability of 25% (same group) / 6.25% (different groups) motivating per-packet entropy values. | Specification overview by the authors; no performance measurements or benchmarks; vendor differentiation leaves many mechanisms optional. | high | full-text | The Ethernet ecosystem's answer to InfiniBand for AI/HPC: defines the modernized high-performance Ethernet transport. |
| metz2024empoweri | survey | Ultra Ethernet Consortium (UEC) and its goals for AI/HPC workloads on Ethernet | qualitative: physical-layer tradeoffs (power, temperature, distance), transport-layer reliability and latency | Discusses the UEC's push to refine the Ethernet stack, particularly the transport layer, to meet AI workloads' scale, latency, and reliability demands; no quantitative results are reported. | Position/discussion paper; abstract-only; no numbers. | moderate | abstract-only | Motivation and consortium context for Ultra Ethernet as the AI interconnect standard. |
| arsid2026ualink | survey | Open-standard interconnect technologies for AI/HPC datacenters: UALink, Ultra Ethernet, and PCIe/CXL (literature survey) | Qualitative bandwidth/latency capabilities and feature comparison | Survey reports UALink targets multi-terabit throughput at sub-microsecond latency in vendor-neutral topologies, Ultra Ethernet adds deterministic forwarding, advanced congestion control, and hardware-accelerated collectives to commodity Ethernet, and CXL enables cache-coherent memory tiering with order-of-magnitude performance gains over software-managed tiering (specific numeric benchmarks not reported). | Secondary source with qualitative claims; no primary experiments or measured benchmarks. | moderate | abstract-only | Positions UALink/Ultra Ethernet/CXL as the emerging hierarchical interconnect stack for scale-out AI, directly relevant to RDMA/Ethernet/RoCE and scale-up fabrics for LLM training clusters. |
| alladi2026impact | simulation | Simulated reconfigurable NIC prioritizing Ultra Ethernet Transport (UET) egress traffic at 200 Gbps with co-located non-UET tenants | Added average latency for non-UET tenants (percent and ns) | Prioritizing UET egress traffic increases average latency for co-located non-UET tenants by less than 2% (under 2 ns absolute at 200 Gbps), a threshold the authors argue is inconsequential. | Simulation-based analysis; single NIC-modification scenario; not validated on real FPGA-based UET NIC implementations. | moderate | abstract-only | Supports incremental adoption of Ultra Ethernet (RoCE-class) transport sharing NICs with legacy tenants in hyperscaler clouds for AI/HPC traffic. |
| hu2025demystif | systems paper | NCCL 2.19.1 source/design analysis plus ATLAHS application-trace-driven network simulator for AI training workloads | Qualitative analysis of protocols (Simple, LL, LL128), transports (NVLink, PCIe, IB verbs/GPUDirect RDMA), ring/tree algorithms | Provides an in-depth analysis of NCCL's communication protocols, channel orchestration, intra/inter-node data movement, and ring/tree collectives, and builds ATLAHS to accurately reproduce NCCL communication patterns at scale (no headline performance numbers reported). | Analysis based on NCCL 2.19.1 so implementation details may evolve; qualitative rather than quantitative benchmarking. | high | full-text | Foundational reference demystifying NCCL internals (protocol variants, GPUDirect RDMA transports, ring/tree algorithms) that underpin large-scale LLM training interconnect stacks. |
| weingram2023survey | survey | Industry-led collective communication libraries (xCCL): NCCL, oneCCL, RCCL, MSCCL, ACCL, Gloo | Design features, use cases, and performance comparison using NCCL Tests and PARAM benchmarks | Surveys design features and industry use cases of six xCCL libraries and compares their performance with the NCCL Tests and PARAM industry benchmarks (specific numeric results not reported in abstract). | Survey-level overview; benchmark numbers not detailed in abstract; library landscape evolves rapidly. | moderate | abstract-only | Maps the collective communication library landscape (NCCL/RCCL/MSCCL/ACCL/Gloo) relevant to interconnect-aware optimization for deep learning workloads. |
| cowan2023mscclang | systems paper | MSCCL++ framework/language for custom collective algorithms on large multi-GPU systems | Qualitative: ability to implement topology- and pattern-specific collectives | Proposes MSCCL++ for implementing correct and efficient custom collective algorithms optimized for both network topology and application-specific communication patterns to alleviate collective bottlenecks in multi-GPU training/serving (no numeric results reported in abstract). | Abstract-only; no performance numbers available locally; requires custom-algorithm implementation expertise. | moderate | abstract-only | Enables topology-aware custom collectives (e.g., for NVLink/InfiniBand topologies), an approach later built upon by synthesis-based schedulers such as TACCL and Canvas. |
| hei2025canvas | systems paper | Collective-communication scheduling synthesis evaluated on topologies up to 512 GPUs; baselines TACCL and TE-CCL | Bandwidth speedup and schedule synthesis time | Canvas achieves up to 1.98x bandwidth speedup over TACCL and 3.56x over TE-CCL, and synthesizes algorithms for 512-GPU topologies within 1.77 hours whereas TACCL fails to produce results within 24 hours. | Evaluated on imbalanced topologies; hierarchical decomposition may trade optimality; cross-micro-batch pipelining adds complexity. | high | abstract-only | Advances scalable, near-optimal topology-aware collective scheduling to overcome vendor-library bandwidth bottlenecks in large GPU clusters used for LLM training. |
| cao2024scheduli | systems paper | 96-GPU testbed experiments plus large-scale production trace-based simulation of multitenant deep-learning-training jobs | GPU computation utilization (%) | Crux, a GPU-intensity-aware communication scheduler, improves GPU computation utilization by 8.3%-14.8% on a 96-GPU testbed and by up to 23% in production-trace simulation versus Sincronia, TACCL, and CASSINI. | Multitenant cloud setting; communication-contention focus; NP-complete problem addressed via heuristic theorem-based approach. | high | abstract-only | Shows communication contention among co-located DLT jobs wastes GPU compute and that prioritizing high-intensity flows mitigates it in shared training clusters. |
| huang2023gzccl | systems paper | Up to 512 NVIDIA A100 GPUs on HPE Slingshot 10 (100 Gbps) interconnect; real-world datasets and image-stacking application | Collective speedup vs NCCL/Cray MPI; reconstructed data quality | gZCCL-accelerated Allreduce and Scatter outperform NCCL by up to 4.5x and Cray MPI by up to 28.7x (gZ-Allreduce ring up to 20.2x over Cray MPI/NCCL), with a 1.69x gain in image stacking while preserving high data quality under a 1E-4 error bound. | Lossy compression accuracy depends on error bounds; compression/decompression still occupies substantial runtime share; evaluated on Slingshot 10-class fabrics. | high | full-text | Demonstrates accuracy-aware lossy compression inside GPU-aware collectives as a way to stretch limited inter-node bandwidth (100 Gbps-class) in large-scale training. |
| li2024optimizi | field study | Operational GPU-centric network managing over 10K heterogeneous GPU cards (production deployment) | Qualitative: collective communication latency, congestion, and utilization of GPU and switch network bandwidth | TCCL jointly optimizes topology-aware collective communication at hosts and centralized routing management in multi-path networks, fully utilizing GPU and switch network bandwidth in parallel and reducing collective delay via short cross-node paths (specific numbers not reported in abstract). | Practice/experience report; quantitative results not given in abstract; single-operator context. | moderate | abstract-only | Field evidence that GPU-centric networks (>10K GPUs) require co-design of collective communication and centralized routing to avoid congestion and jitter in LLM training. |
| liu2026architec | systems paper | UBEP evaluated on up to 256 NPU dies from a production Huawei CM384 superpod (Unified Bus fabric, ~400 GB/s, hundreds-of-ns latency); MoE inference | All-to-All latency reduction and MoE inference Time Per Output Token (TPOT) | UBEP reduces All-to-All latency by up to 52.4% and MoE inference TPOT by up to 11.1% versus the baseline CANN EP, where all-to-all communication consumes roughly 47% of total execution time. | Evaluated on Huawei CANN/NPU stack only; analysis specific to superpod scale-up fabrics (NVL72/576, CloudMatrix384) rather than traditional IB/RoCE clusters. | high | full-text | Quantifies how BSP serialization, synchronization overhead, and distance-agnostic scheduling waste high-bandwidth scale-up fabrics (UB/NVLink-class) in MoE all-to-all traffic. |
| goldman2026unified | systems paper | NCCL EP library evaluated on an H100-based cluster across 1-8 node configurations; vLLM and Megatron-LM integration; NVLink intra-node + RDMA (GIN) inter-node | Low-latency kernel latency (decode, 1-128 tokens), high-throughput kernel throughput (4096+ tokens), end-to-end vLLM results | NCCL EP provides unified ncclEpDispatch/ncclEpCombine primitives on NCCL's Device API with LL mode (1-128 tokens) and HT mode (4096+ tokens), achieving competitive LL kernel performance versus DeepEP and end-to-end results with vLLM (specific latency/throughput numbers not reported in read portions). | Specific numeric results not captured in the portions read; evaluation limited to H100 clusters; requires NCCL 2.28+ Device API/GIN support. | moderate | full-text | Brings MoE dispatch/combine (fine-grained all-to-all over RDMA+NVLink) natively into NCCL, unifying device-initiated expert-parallel communication for training and inference. |
| zheng2026ncclbpf | systems paper | 8x NVIDIA B300 GPUs connected via NVLink; NCCL 2.29.7 with bpftime userspace eBPF runtime | Tuner-decision overhead (ns), AllReduce throughput, policy hot-reload downtime | NCCLbpf adds 80-130 ns overhead per tuner decision (less than 0.03% of collective latency), hot-reloads policies atomically in 1.07 us with zero dropped calls across 400,000 invocations, and improves 8-GPU AllReduce throughput by up to 27% over NCCL defaults in the 4-128 MiB range. | Single 8-GPU NVLink-node evaluation; selects among NCCL built-in algorithms only (no custom algorithms); userspace eBPF verification scope. | high | full-text | Demonstrates verified, composable, hot-reloadable policy extension (eBPF-style) for NCCL's plugin interface, improving collective throughput without modifying NCCL source. |
| jia2026inspecto | case study | Nemotron-4 LLM pretraining (340B model) on NVIDIA H100 clusters with 16 to 2,048 GPUs; NVLink/PCIe intra-node and InfiniBand/RoCE inter-node | NCCL Inspector metrics: algorithmic/hardware bus bandwidth, latency, message size per collective type | NIXT converts NCCL Inspector profiling into accessible analysis, attributing straggler anomalies and showing how communication phases change with ML parallelism and GPU scale during Nemotron-4 pretraining up to 2,048 GPUs, where collective communication commonly consumes 20-40% of training time. | Case study on one model family and cluster; observability tooling rather than an optimization itself. | high | full-text | Provides production-scale observability into collective communication (bandwidth, stragglers, phase attribution) needed to diagnose interconnect bottlenecks in LLM pretraining. |
| shen2026every | systems paper | NCCL device-side low-latency collective kernels on GPU scale-up networks; Llama-3.1-70B tensor-parallel inference; cuSOLVERMp; microbenchmarks vs speed-of-light bound | Small-message AllReduce latency (us), inter-token latency (ITL), cost per 1M output tokens | New symmetric NCCL kernels reduce small-message AllReduce latency from 11.0 us (NCCL ring) to 2.37 us, within 7% of the hardware speed-of-light lower bound, yielding an 8.7% ITL reduction for Llama-3.1-70B inference and about 0.9% cost saving per microsecond removed from AllReduce latency. | Latency-focused optimizations for small/medium messages; scale-up (single-node NVLink-class) networks; bandwidth-bound large messages benefit less. | high | full-text | Shows latency, not just bandwidth, dominates small collectives on decode-heavy LLM inference critical paths, and quantifies microsecond-level gains achievable on scale-up interconnects. |
| singh2025scalable | systems paper | PCCL evaluated on Frontier (2,048 AMD MI250X GCDs) and Perlmutter (NVIDIA A100); DeepSpeed ZeRO-3 and DDP training workloads | All-gather/reduce-scatter/all-reduce speedup; end-to-end training throughput speedup | PCCL achieves up to 168x faster reduce-scatter, 33x faster all-gather, and 10x faster all-reduce than RCCL on 2,048 Frontier GCDs (up to 5.7x over NCCL on Perlmutter), translating to up to 4.9x DeepSpeed ZeRO-3 and 2.4x DDP training speedups. | Optimized for large messages (>10 MB); hierarchical learning-based algorithm selection may not generalize to all topologies; vendor-specific tuning. | high | full-text | Quantifies large headroom in vendor collective libraries (RCCL/NCCL) at supercomputer scale, informing interconnect-aware collective design for large-buffer DL workloads. |
| zhang2025reliable | systems paper | Production GPU training clusters (largest deployment 24K GPUs); 8-GPU Hopper + 8 ConnectX-7 RNIC testbed with 400 Gbps 1:1 two-tier CLOS | Training throughput, inter-node small-message latency, GPU time wastage under RNIC failures | VCCL improves training throughput by up to 5.28% over NCCL, reduces inter-node small-message latency by 18.9% on average, and reduces GPU time wastage by nearly 90% during NIC port failures via a primary-backup QP mechanism. | Single-company deployment experience; healthy-path gains are percent-level; requires multi-NIC servers and RDMA monitoring support. | high | full-text | Production evidence that SM competition, RNIC port failures, and transient RDMA anomalies waste GPU resources, and that SM-free P2P plus fault-tolerant QPs recover most of the loss. |
| zhang2023grained | systems paper | 64-GPU cluster with 10Gb/s Ethernet and 100Gb/s InfiniBand interconnects; five popular DNN models | Training speedup versus state-of-the-art all-reduce scheduling | DeAR decouples all-reduce into two continuous operations overlapped with both backpropagation and feed-forward, achieving up to 83% training speedup on 10Gb/s Ethernet and 15% on 100Gb/s InfiniBand, reaching 72.3-99.2% of the maximum possible speedup. | Evaluated at 64-GPU scale; gains depend on interconnect latency/bandwidth regime; scheduling complexity grows with model size. | high | full-text | Shows fine-grained overlap of decoupled all-reduce with both backward and forward passes, with larger gains on low-bandwidth/high-latency Ethernet than on InfiniBand. |
| wang2023extremel | systems paper | GPT-3-like models (10-138B parameters) trained on up to 384 GPUs; low-bandwidth clusters with cross-node links often below 100 Gbps Ethernet | Communication volume reduction, throughput speedup, sustained peak throughput | ZeRO++ reduces ZeRO communication volume 4x (block-quantized all-gather cutting weight gathering 50%, hierarchical remapping eliminating inter-node backward all-gather, all-to-all quantized gradient reduction cutting gradients 75%), enabling up to 2.16x better throughput at 384 GPUs, over 45% sustained peak throughput, and up to 2.4x end-to-end speedup over ZeRO. | Lossy quantization needs convergence checks; hierarchical partitioning trades memory for communication; evaluated up to 384 GPUs. | high | full-text | Communication-volume reduction techniques that make ZeRO-3 viable on bandwidth-limited Ethernet clusters, a key regime for LLM training outside InfiniBand/NVLink datacenters. |
| park2026fully | systems paper | Commodity (bandwidth-limited) GPU cluster; ZeRO-3 fully sharded data parallel training and parameter-efficient fine-tuning (PEFT) | Inter-node all-gather traffic reduction and training throughput | FCDP caches forward-pass parameters in host memory for fast intra-node reuse, reducing inter-node all-gather by 50% (over 99% for PEFT) and achieving up to 100x higher throughput than ZeRO-3 and 51x over ZeRO++ while keeping ZeRO-3's minimal GPU memory footprint. | Assumes host-memory bandwidth beats inter-node links; PEFT-specific gains rely on selective communication; commodity-cluster focus. | high | abstract-only | Reframes host memory as a fast caching layer to bypass slow inter-node interconnects, a practical trick for LLM training on non-NVLink/InfiniBand commodity fabrics. |
| xu2026lagom | systems paper | High- and low-bandwidth GPU clusters; diverse models and parallelization strategies; baselines NCCL and AutoCCL | Training speedup from communication-computation overlap tuning | Lagom co-tunes communication parameters with a unified cost model and priority-based search (reducing complexity from exponential to linear), achieving 1.07-1.33x speedup over NCCL and 1.03-1.27x over AutoCCL, where mis-tuned communication contention degrades bottlenecked computation by up to 35%. | Tuning scope limited to NCCL-style communication parameters; search heuristic may miss global optimum; per-cluster calibration needed. | high | full-text | Shows communication tuning must also target computation-bottlenecked regimes, informing overlap strategies on both RDMA and NVLink fabrics in LLM training. |
| jia2024energy | systems paper | Characterization of NCCL collective operations plus PCCL DVFS-based power management on GPU clusters; end-to-end LLM training | Energy consumption of collective communication and end-to-end training energy | PCCL identifies the optimal GPU frequency per collective call and reduces collective communication energy by roughly 27% and end-to-end LLM training energy by 17.3% with negligible throughput impact, where collectives often account for more than half of training time. | DVFS benefits depend on workload and GPU model; power measurement granularity; interconnect specifics (NVLink/InfiniBand) not detailed in abstract. | moderate | abstract-only | Adds an energy dimension to interconnect/collective design: communication kernels dominate training time and offer DVFS-based power savings with minimal performance cost. |
| jin2024software | systems paper | SDCC software-defined collective communication framework; PyTorch and TensorFlow engines; Linux TCP and kernel-bypass transports | Qualitative: flexibility, generality, and performance via dataflow abstraction | SDCC decouples control plane from data plane using a unified dataflow graph, enabling users to express new collective algorithms, integrate multiple compute engines and network transports (TCP and kernel bypass), and exploit parallelism, demonstrated in four use cases (specific numbers not reported in abstract). | Abstract-only; no quantitative results available locally; general framework may add overhead versus hand-tuned vendor libraries. | moderate | abstract-only | Software-defined, transport-agnostic collectives (including kernel-bypass/RDMA-class transports) as an alternative to tightly coupled vendor communication stacks. |
| li2024understa | benchmark | Systematic empirical characterization of distributed training communication across determinant factors (parallelism, models, clusters) | Communication pattern features (e.g., predictability) and communication overhead factors | Provides a systematic analysis of communication patterns and overhead determinants and develops a validated analytical formulation to estimate communication overhead for models exhibiting predictability (specific numeric findings not reported in abstract). | Abstract-only; characterization may be tied to specific frameworks and cluster configurations. | moderate | abstract-only | Empirical basis for predicting communication overhead in distributed LLM training, informing when interconnect optimizations (RDMA, collectives) matter most. |
| samsi2020benchmar | benchmark | Data-parallel deep learning training on shared HPC systems comparing Ethernet and OmniPath fabrics with GPUDirect and NCCL; also CFD workloads | Impact of interconnect hardware and network software primitives on training time | Using Ethernet-based networking in shared HPC systems does not have a significant effect on training times for commonly used DNN architectures or traditional HPC applications such as CFD compared with OmniPath (specific numbers not reported in abstract). | Shared-system noise; older hardware generation; data-parallel workloads only. | moderate | abstract-only | Benchmark evidence that for modest data-parallel workloads interconnect choice (Ethernet vs OmniPath) matters little, contrasting with today's bandwidth-hungry LLM training. |
| chen2026failures | theoretical | Information-theoretic analysis of AllReduce under asymmetric bandwidth; OptCC validated in SimAI simulations of practical failures with up to 50% bandwidth loss; H100/B100-class multi-NIC servers | AllReduce completion time overhead relative to fault-free optimum | Derives the first information-theoretic lower bound on AllReduce time under asymmetric bandwidth (unavoidable overhead only O(1/p) when the straggler retains at least half its bandwidth) and OptCC completes AllReduce within 2-6% of NCCL's fault-free ring performance, versus up to 57% overhead for the state of the art. | Validation is simulation-based (SimAI); bound requires the straggler to retain at least half bandwidth; primarily single-straggler analysis. | high | full-text | Quantifies and nearly eliminates the AllReduce penalty of NIC-failure rerouting (e.g., PXN-style failover) that otherwise throttles entire data-parallel training collectives. |
| wang2025reliable | systems paper | Two 8-GPU H100 InfiniBand servers plus large-scale ML simulators modeling hundreds of GPUs with diverse NIC failure patterns | Training/inference overhead under failures and failover robustness versus baselines | R2CCL provides lossless, low-overhead multi-NIC failover with less than 1% training and less than 3% inference overhead, outperforming AdapCC by 12.18x and DejaVu by 47x, in a context where network faults can waste 10-15% of GPU hours. | Physical testbed is small (two servers); large-scale results come from simulation; requires multi-NIC hardware. | high | abstract-only | Fault-tolerant collective library exploiting multi-NIC RDMA connection migration to prevent job-killing timeouts and checkpoint rollbacks in InfiniBand-based training clusters. |
| pan2025comprehe | survey | GPU collective deadlock phenomena, causes, detection, and prevention in distributed deep learning | Qualitative taxonomy of deadlock scenarios and available solutions | Surveys GPU collective deadlocks that arise when multiple collectives circularly wait for each other and concludes only ad-hoc consistent-ordering methods exist for specific scenarios, with no general effective solution (specific numbers not reported). | Survey-level treatment; no quantitative measurements; fast-moving field. | moderate | abstract-only | Catalogues correctness hazards (deadlocks) in collective communication stacks that practitioners must handle when orchestrating interconnect-heavy LLM training. |
| arzani2023rethinki | theoretical | TE-CCL evaluated on GPU topologies including Azure NDv2 (2-chassis) and larger topologies; compared with TACCL, SCCL, and heuristics; BERT/DeepLight idle-time context | Schedule quality (finish time / bytes sent) and solver scalability | TE-CCL formulates collective scheduling as a scalable multi-commodity flow MILP, improving TACCL's solution quality by a minimum of 2x in many scenarios while scaling to larger topologies where SMT-based approaches fail, in a context where GPUs idle 11% (BERT) and 63% (DeepLight) without good schedulers. | MILP/LP relaxations may sacrifice optimality on some inputs; evaluation on specific topologies; headline is schedule quality, not measured runtime. | high | full-text | Recasts ML collective scheduling as a traffic-engineering-style flow problem, producing scalable, higher-quality schedules for AllToAll/AllGather/AllReduce on GPU fabrics. |
| hamidouch2025initiate | systems paper | NCCL 2.28 Device API GPU-Initiated Networking (GIN) architecture; DOCA GPUNetIO kernel-initiated (GDAKI) and Proxy backends; DeepEP integration on multi-node GPU clusters | Qualitative: operation latency and CPU offload; point-to-point microbenchmarks and DeepEP application-level benchmarks | GIN enables device-initiated RDMA within NCCL's unified runtime via kernel-initiated and proxy backends, eliminating CPU orchestration for MoE dispatch/combine and demonstrating practical integration with DeepEP (specific numeric results not reported in read portions). | Requires NCCL 2.28+ and ConnectX-6 Dx or newer NICs for the GDAKI path; proxy backend trades latency for compatibility; specific numbers not captured in read portions. | moderate | full-text | Documents the hardware/software path for GPU-initiated RDMA (GPUDirect Async / DOCA GPUNetIO), central to low-latency MoE all-to-all on InfiniBand/RoCE fabrics. |
| hwang2022tutel | systems paper | Flex (Tutel) MoE stack on Azure A100 clusters from 16 to 2,048 GPUs; SwinV2-MoE vision model compared with Fairseq | MoE layer speedup and end-to-end training/inference speedup | Flex delivers 4.96x (16 GPUs) and 5.75x (2,048 GPUs) single-MoE-layer speedups over the prior state of the art (up to 3.11x at 128 GPUs) and 1.55x/2.11x end-to-end training/inference speedups over Fairseq, using flexible 2D hierarchical all-to-all and zero-cost adaptive parallelism/pipelining. | Vision-MoE focus; speedups depend on token-routing imbalance; requires identical parameter/data layout design. | high | full-text | Adaptive parallelism and hierarchical all-to-all for MoE, addressing dynamic token-routing traffic patterns on A100 clusters relevant to LLM-scale expert parallelism. |
| rajbhanda2022deepspee | systems paper | DeepSpeed-MoE training and inference on 128 NVIDIA A100 GPUs (Azure ND A100); 1.3B+MoE-128 vs 6.7B dense; 300B training tokens | Training throughput, inference latency/cost, model size reduction | DeepSpeed-MoE achieves a 5x training compute reduction (372 vs 70 training samples/sec on 128 A100s), compresses MoE model size by up to 3.7x, and delivers 7.3x better inference latency and cost than existing MoE solutions and up to 4.5x faster, 9x cheaper inference than quality-equivalent dense models. | Gains are architecture-level rather than interconnect-specific; inference noted as memory-bandwidth bound; dense-baseline comparisons are model-specific. | high | full-text | Establishes MoE all-to-all communication as a first-class cost in large-model training/inference systems, motivating interconnect- and RDMA-aware expert-parallel design. |
| deepseeka2024deepseek | systems paper | DeepSeek-V2 MoE LLM (236B total / 21B active parameters), 128K context, pretrained on 8.1T tokens | Training cost, KV cache size, generation throughput | Compared with DeepSeek 67B, DeepSeek-V2 saves 42.5% of training costs, reduces KV cache by 93.3%, and boosts maximum generation throughput to 5.76x, using Multi-head Latent Attention and DeepSeekMoE with device-limited routing to bound MoE-related communication costs. | Model technical report rather than interconnect study; communication control is architectural (routing); cluster hardware details limited. | high | full-text | Production MoE architecture whose device-limited routing and communication-balance losses directly shape all-to-all traffic on GPU interconnects during training and inference. |
| dai2024deepseek | systems paper | Not reported - the local abstract record contains citation metadata only (Dai et al., ACL 2024, DeepSeekMoE paper) | Not reported | Not reported - the local abstract file contains only bibliographic metadata, so no quantitative findings are available locally. | Abstract record is citation-only; no text or numbers available in the local corpus. | low | abstract-only | Citation-only record of the DeepSeekMoE architecture paper; relevant as MoE routing/expert-specialization context for all-to-all communication in LLM training. |
| deepseeka2024deepseek2 | systems paper | DeepSeek-V3 MoE LLM (671B total / 37B active parameters) pretrained on 14.8T tokens on a 2048-H800-GPU cluster with InfiniBand and NVLink | Training cost (H800 GPU hours), all-to-all communication overhead, pipeline efficiency | DeepSeek-V3's full training costs only 2.788M H800 GPU hours (180K H800 GPU hours per trillion tokens, i.e., 3.7 days per trillion on 2048 H800s), using DualPipe computation-communication overlap and InfiniBand/NVLink-optimized cross-node all-to-all kernels to achieve near-zero all-to-all communication overhead. | Single-cluster, single-vendor hardware report; numbers specific to H800 plus the proprietary IB/NVLink topology; generalizability to other fabrics unproven. | high | full-text | Flagship evidence that computation-communication overlap plus IB/NVLink-optimized all-to-all kernels make 671B-scale MoE training communication-near-free and cost-effective. |
| nie2024locality | systems paper | RoBERTa-MoE, GPT-MoE, and Swin-MoE training on four A100 servers with 200 Gb/s cross-machine bandwidth | All-to-all communication ratio of training time; training speedup | All-to-all communication averages 45% (up to 67%) of MoE training time, and LSH-MoE's locality-sensitive-hashing token compression with residual error compensation achieves 1.28x-2.2x training speedups across language and vision MoE models. | Lossy compression requires error compensation; evaluated at four-node scale; gains depend on token similarity in the workload. | high | full-text | Quantifies all-to-all as the dominant cost in MoE training (45%+ of time on 200 Gb/s fabrics) and shows token-similarity compression as a communication-volume mitigation. |
| zhang2026hiding | systems paper | MoE training (OLMoE, DeepSeek-MoE-16B) under expert parallelism with all-to-all communication | MoE layer execution time / training efficiency; all-to-all communication hiding | HALO reduces execution time on OLMoE by 26.1%, 22.9%, and 22.0% on average versus PipeMoE, MPMoE, and ScMoE respectively, and by 15.2%, 13.1%, and 20.4% on DeepSeek-MoE-16B. | Shadow-expert replication adds memory overhead; benefit hinges on routed-token locality and a sufficiently large local computation window. | high | abstract-only | Shows all-to-all communication in expert-parallel MoE training can be hidden via locality-aware overlap and shadow experts, a key technique for interconnect-bound LLM training. |
| qi2026feplb | systems paper | GLM-5 MoE layers (128 experts, no auxiliary loss, up to 16 H100 GPUs, EP up to 8) | Token/GEMM straggler, EP communication overhead, GPU time waste | FEPLB reduces the token straggler by 51-70% and the GEMM straggler by 50-68% with no measurable EP communication overhead on GLM-5's MoE layers, and at EP=8 achieves 2x lower token straggler than FasterMoE, where load imbalance alone wastes on average 18.6% of GPU time per MoE layer. | Requires NVIDIA Hopper NVLink Copy Engine and only rebalances within the intra-node NVLink domain; evaluated on a single model family. | high | full-text | Exploits the NVLink Copy Engine as a near-free intra-node communication channel parallel to compute kernels, informing NVLink usage for MoE load balancing in LLM training. |
| qin2026reducing | systems paper | MoE model training with hybrid parallelism across Pods in multi-tenant NPU clusters (models/topologies not specified in abstract) | Cross-Pod traffic volume, communication time | A two-stage Best-Fit + neighborhood-search placement scheme for MoE hybrid-parallel training reduces cross-Pod traffic by 35.9% and cuts communication time by 18.7% compared to state-of-the-art methods. | Abstract-only record; NPU- and multi-tenant-specific; relies on quantifying cross-Pod communication overhead as a 0-1 integer quadratic problem. | moderate | abstract-only | Addresses cross-Pod placement to mitigate intra-/inter-Pod bandwidth and latency discrepancies for MoE training, relevant to interconnect-aware job placement. |
| zhang2026dynamic | systems paper | MoE LLMs (e.g., DeepSeek-v3) on multi-GPU NVLink/NVSwitch systems; simulated GH200 NVL32 | End-to-end and MoE-layer speedup, redundant traffic reduction | DySHARP achieves up to 1.79x speedup over the state of the art via dynamic in-switch computing (NVLink SHARP extension), where MoE communication consumes 50-80% of execution time and profiling shows near 50% communication redundancy. | Requires ISA, microarchitecture, and switch changes (hardware co-design); evaluation uses simulated GH200 NVL32 hardware. | high | full-text | Extends NVLink SHARP in-switch reduction to irregular dynamic MoE traffic, showing in-switch computing can cut redundant dispatch/combine transfers in LLM training. |
| zhu2025megascal | systems paper | Not reported (partial abstract; MoE LLM inference context) | GPU utilization, operational cost (motivational only) | Not reported - the partial abstract only motivates that MoE's sparsely activated architecture shifts FFNs from compute-intensive to memory-intensive during inference, lowering GPU utilization and raising cost. | Abstract is truncated/partial with no methods or results; full text unavailable. | low | abstract-only | Partial abstract frames memory-bound MoE inference efficiency, relevant to memory/interconnect pressure in LLM serving. |
| cai2024shortcut | systems paper | SwinV2-MoE-S vision and language MoE models; 8x A30 and 8x A800 GPU nodes | Training/inference speedup, All-to-All communication share, overlap ratio | ScMoE achieves 1.49x training and 1.82x inference speedups over a top-2 MoE baseline with up to 100% communication-computation overlap, where All-to-All takes 60% of block time on 8x A30 but drops to 15% on 8x A800 due to ~6x higher NVLink bandwidth. | Changes the MoE architecture (shortcut connections), which may affect gating/model quality; inter-node Ethernet still limits multi-node scaling. | high | full-text | Quantifies how NVLink bandwidth (A30 vs A800) changes All-to-All overhead and shows shortcut-connected MoE decouples communication from computation for overlap. |
| wang2022topoopt | systems paper | 12-node direct-connect prototype with RDMA forwarding at 100 Gbps; large-scale simulations of real distributed training models | DNN training time, AllReduce and model-parallel transfer efficiency | TopoOpt reduces DNN training time by up to 3.4x versus similar-cost Fat-Tree interconnects in large-scale simulations, validated on a fully functional 12-node direct-connect prototype with RDMA forwarding at 100 Gbps. | Requires direct-connect (optical circuit switch) fabric and host-based RDMA forwarding via NIC NPAR; large-scale results are simulation-based. | high | full-text | Pioneers co-optimizing network topology with AllReduce traffic mutability for DNN training, foundational for topology-aware interconnect design (RDMA/OCS). |
| namkoong2025sortingh | systems paper | Various LLMs on diverse multi-GPU environments (topology-aware) | Inference latency, speedup over single-GPU baseline | SortingHat, using dominator analysis plus MILP-based scheduling that exploits both model parallelism and data-transfer parallelism, achieves an average speedup of 2.28x and up to 2.96x over the single-GPU TVM baseline. | MILP-based search may not scale to very large models; inference-only focus; baselines limited to TVM. | moderate | abstract-only | Shows multi-GPU LLM inference gains depend on co-scheduling data-transfer parallelism with system topology, relevant to interconnect-aware scheduling. |
| chang2026spiderfl | systems paper | Not reported (citation-only record) | Not reported | Not reported - only bibliographic metadata (ACL 2026) is available; the abstract file contains no content. | Record contains citation metadata only; no abstract or full text to extract evidence from; design inferred from title. | low | abstract-only | Citation-only record (SpiderFL, ACL 2026); likely federated-learning communication/scheduling work but no extractable interconnect evidence. |
| ye2023workload | survey | Published DL scheduling research for training and inference workloads in GPU datacenters | Scheduling objectives, resource utilization, throughput (qualitative) | Survey of DL workload scheduling in GPU datacenters covering training and inference schedulers, objectives, and resource-utilization mechanisms; no single quantitative result (survey). | Survey-level; no new measurements; specific numbers not summarized in the abstract. | high | abstract-only | Provides a taxonomy of GPU-datacenter scheduling including communication- and resource-aware methods, context for interconnect-aware training scheduling. |
| yu2022scheduli | theoretical | Analytical model plus experiments on GPU clusters with multiple ring-all-reduce training jobs (details in full text) | Makespan of co-located ring-all-reduce training jobs | Not reported in abstract - proposes an analytical model of RAR communication overhead and contention and an SJF-BCO (Smallest Job First with Balanced Contention and Overhead) approximation algorithm to minimize makespan. | Abstract truncated; theoretical results may rely on simplifying contention assumptions. | moderate | abstract-only | Models communication contention between co-located all-reduce training jobs, relevant to RDMA/collective scheduling in GPU clusters. |
| dong2026phase | systems paper | LLM training communication phases on OCS-reconfigurable optical networks (scale not stated) | Communication time over static vs reconfigurable optical networks | A phase-aware optical network using OCSs to reconfigure into phase-optimal topologies with Rabenseifner/Bruck mappings and slot amortisation achieves 37.5% faster communication than static optical networks under established LLM training configurations. | Short OFC abstract; evaluation methodology (simulation/testbed) and scale not stated. | moderate | abstract-only | Evidence that reconfigurable optical (OCS) topologies can accelerate LLM training collectives by matching topology to the communication phase. |
| cho2023logical | systems paper | 8-GPU NVIDIA DGX-1 proof-of-concept | AllReduce communication time, end-to-end training time | C-Cube chains and overlaps tree-AllReduce phases with forward computation via gradient queuing and adds physical-topology detour/double-tree variants, improving communication and overall performance versus non-overlapped tree algorithms (magnitude not reported in abstract). | Abstract gives no numeric speedup; prototype limited to 8 GPUs. | moderate | abstract-only | Demonstrates logical/physical topology-aware AllReduce with computation chaining on an NVLink-based DGX-1, relevant to collective optimization. |
| won2023astra | simulation | Simulated case studies of emerging distributed training models/platforms (wafer-scale nodes, multi-dimensional topologies, disaggregated memory) | Training throughput/time via analytical performance estimation | Not reported - ASTRA-sim2.0 is a simulation infrastructure extension supporting arbitrary model parallelization, parameterizable multi-dimensional heterogeneous topologies, and in-network collective/memory modeling for design-space exploration at scale. | Simulation-only; accuracy depends on analytical performance models. | high | abstract-only | Provides an open simulator for interconnect/topology-aware distributed training design-space exploration (NVLink, InfiniBand, collectives, disaggregated memory). |
| jiang2024megascal | field study | 175B-parameter LLM training on 12,288 GPUs (production system at >10,000-GPU scale) | Model FLOPs Utilization (MFU), training stability, fault tolerance | MegaScale achieves 55.2% MFU when training a 175B LLM on 12,288 GPUs, improving MFU by 1.34x compared to Megatron-LM. | Single proprietary stack; MFU depends on specific model/hardware; stability fixes are largely empirical. | high | full-text | Production evidence that communication-computation overlap, network tuning, and observability sustain 55% MFU at 12k+ GPUs - a reference point for interconnect-bound LLM training. |
| meng2025astral | field study | Production LLM training/inference datacenter infrastructure deployed over 18 months for multiple customers | Interconnect architecture (same-rail tier-2), failure diagnosis, operator timeline forecasting | Not reported in abstract - Astral deploys a same-rail interconnection network on tier-2 for LLM training with full-stack monitoring and operator-granular Seer forecasting for fault diagnosis and network upgrades. | No quantitative results in the abstract; deployment-specific insights. | moderate | abstract-only | Field experience with same-rail network design for LLM training clusters, relevant to datacenter interconnect architecture. |
| jouppi2023opticall | field study | TPU v4 supercomputer, 4096 chips, production since 2020 | Achieved FLOPS fraction, relative performance/power, OCS cost and power share | TPU v4 with OCS-reconfigurable twisted 3D torus trains a large language model at an average of ~60% of peak FLOPS, outperforms TPU v3 by 2.1x (2.7x per Watt) and A100 by 1.2-1.7x, while OCSes cost <5% of system cost and <3% of system power. | Google-specific deployment; OCS reconfiguration overheads and topology choices not fully quantified here. | high | abstract-only | Seminal evidence that OCS-based reconfigurable optical interconnects are cheaper and lower power than InfiniBand and sustain ~60% peak FLOPS for LLM training. |
| liu2023lightwav | field study | 4096-chip TPU v4 ML superpod (>1 ExaFLOP) with reconfigurable lightwave fabric | System availability, model training performance, cost share | Lightwave fabric (OCS + WDM/circulators) deployment provides up to 3x better system availability and model-dependent performance improvements of up to 3.3x versus a static fabric, at less than 6% of total system cost. | Google-internal deployment; performance gains are model-dependent. | high | abstract-only | Shows reconfigurable optical lightwave fabrics improve availability and LLM training performance, supporting OCS-based interconnect arguments. |
| shou2025infinite | systems paper | High-bandwidth domain (HBD) architectures for LLM training (NVL-72, TPUv3, Dojo, TPUv4) | Scalability, cost, fault resiliency of HBD interconnects (qualitative) | Not reported - partial abstract only surveys HBD architectures (switch-centric NVL-72, GPU-centric TPUv3/Dojo, hybrid TPUv4) and their scalability, cost, and fault-propagation tradeoffs. | Truncated abstract; no results available. | low | abstract-only | Frames the high-bandwidth-domain (NVLink-domain) scalability/cost/fault tradeoff space for LLM training interconnects. |
| feng2025mercury | systems paper | FPGA-based real-time prototype of an optical-switched network for ML training | Reconfiguration granularity, training acceleration | Mercury, combining collaborative optical timeslot and circuit switching, demonstrates nanosecond-granularity reconfiguration and model-training acceleration on a real-time FPGA prototype (speedup magnitude not reported in abstract). | Short OFC abstract; acceleration magnitude and scale not reported. | moderate | abstract-only | Evidence for nanosecond-scale reconfigurable optical switching as an interconnect option for ML training. |
| yang2025multi | systems paper | Multi-GPU systems; Rodinia benchmarks and LLM training kernels | Speedup, energy consumption, chip area | BEAM, a multi-channel in-package optical interconnect architecture for multi-GPU systems, achieves a speedup of 1.14-1.9x and reduces energy by 29-44% compared to electrical-interconnected systems and state-of-the-art schemes, with comparable chip area. | Architecture-level evaluation; optical packaging/thermal realities not covered in the abstract. | moderate | abstract-only | Argues for in-package optical (vs electrical) GPU interconnects for bandwidth and power at scale, relevant to NVLink-alternative roadmaps. |
| zhou2026packaged | survey | Articles in the APNexus co-packaged optics (CPO) theme issue | Not applicable (editorial overview) | Not reported - editorial overview by APNexus editors presenting the articles in the CPO theme issue. | Editorial content only; no data or measurements. | low | abstract-only | Points to co-packaged optics as a packaging-level interconnect direction for AI/network systems. |
| nabavinej2020overview | survey | Published DNN accelerator interconnection designs (ASIC and non-ASIC, NoC, reconfigurable, in/near-memory) | On-chip bandwidth, power, computing flexibility (qualitative) | Not reported - comprehensive survey of on-chip interconnection methods for DNN accelerators covering different interconnection methods, reconfigurable DNN computing, and emerging in/near-memory processing technologies. | Survey-level; no new quantitative measurements. | high | abstract-only | Background survey on on-chip NoC/interconnect design tradeoffs for DNN accelerators, upstream of scale-out interconnects like NVLink/RoCE. |
| jung2025compute | theoretical | Large-scale AI hardware and data center designs (GB200/Blackwell, hierarchical clusters); parametric evaluations | Communication overhead share of training time, memory/interconnect bandwidth (TB/s, GB/s, Gb/s) | Technical report citing that inter-GPU communication accounts for 35-70% of total training time in large-scale AI deployments, describing GB200 with 192 GB HBM3e at up to 8 TB/s per GPU and ~900 GB/s NVLink C2C to 480 GB LPDDR5X, and proposing CXL-over-XLink (UALink/NVLink) disaggregation to reduce long-distance transfers. | Position/report piece; many figures cited from industry analyses; proposed architecture not fully validated in production. | moderate | full-text | Useful synthesis of the communication-tax (35-70% of training time) and the NVLink/CXL/UALink bandwidth hierarchy for LLM infrastructure. |
| gouk2023memory | systems paper | Host processor complex + remote memory over CXL.mem vs RDMA (diverse real-world workloads) | Memory access performance (relative to RDMA-based pooling) | DirectCXL, which directly connects a host processor to remote memory over CXL.mem, exhibits around 7x better performance than RDMA-based memory pooling across diverse real-world workloads. | Prototype-based empirical study; CXL hardware ecosystem still nascent at publication. | moderate | abstract-only | Quantifies CXL.mem vs RDMA for memory disaggregation, relevant to KV-cache/memory pooling for LLM serving. |
| gu2025enabled | systems paper | LLM inference workloads (autoregressive decoding, KV cache, context windows up to 1M tokens) | Memory bandwidth demand vs compute throughput (qualitative) | Not reported - abstract motivates that LLM inference is memory-bandwidth-bound due to low operational intensity and large per-prompt KV caches, and that compute-throughput-optimized GPUs/TPUs are poorly utilized for such tasks. | Partial/motivational abstract; no methods or results. | low | abstract-only | Frames memory-bandwidth-bound LLM inference as the driver for high-bandwidth memory and interconnect systems. |
| ko2026internal | systems paper | CXL memory expander workloads (cold/hot page mixes) | Effective memory capacity, access speedup | IBEX, a block-level compression scheme for CXL memory expanders that compresses only cold data with internal bandwidth-efficient block management, achieves average 1.28x-1.40x speedups over state-of-the-art promotion-based block-level approaches. | Compression adds overhead for cold-data access; compared against compression baselines rather than full systems. | moderate | abstract-only | Addresses bandwidth/capacity limits of CXL memory expanders, an emerging tier for LLM KV-cache offload. |
| quinn2025compute | systems paper | CXL memory expanders with near-memory accelerators for dense retrieval (RAG) | Dense retrieval acceleration, memory access interference (qualitative) | Not reported in abstract - IKS, a compute-enabled CXL memory expander with lightweight near-memory accelerators leveraging high internal bandwidth, accelerates dense retrieval for RAG while supporting spatial and temporal multi-tenancy. | No quantitative results in the abstract. | moderate | abstract-only | Combines CXL memory expansion with near-memory compute for RAG retrieval, relevant to memory-centric LLM inference. |
| fang2025dynamic | theoretical | Heterogeneous memory (HBM + LPDDR5X/NVLink) LLM inference systems; simulation-based validation | Aggregated bandwidth utilization, inference throughput upper bound | A formal treatment of dynamic KV-cache placement across HBM and off-package DRAM (NVLink/LPDDR5X) derives a theoretical upper bound achieving up to 5.87x higher throughput than static placement in simulation, with NVLink 4.0 at up to 100 GB/s per link and GH200 NVLink-C2C at ~900 GB/s. | Theoretical bound, not a deployed policy; assumes knowledge of future access patterns. | moderate | full-text | Quantifies headroom in KV-cache placement across HBM/off-package DRAM tiers, informing memory-interconnect design for LLM inference. |
| wang2022enabling | systems paper | Distributed DL training with disaggregated memory (cache-coherent interconnect + MPI-like collectives) | Training time vs state-of-the-art MPI AllReduce | COARSE, a disaggregated memory extension built on cache-coherent interconnect protocols with decentralized parameter synchronization and dynamic tensor routing, achieves up to 48.3% faster DL training than state-of-the-art MPI AllReduce communication. | Requires CCI-capable hardware; evaluation scope (cluster size, models) not stated in abstract. | moderate | abstract-only | Shows cache-coherent interconnects plus disaggregated memory can beat MPI AllReduce for parameter synchronization, relevant to RDMA/CCI tradeoffs. |
| li2023memory | systems paper | 158 cloud production workloads; CXL memory pools spanning 8-16 sockets | DRAM cost reduction, performance relative to same-NUMA-node allocation | Pond, a CXL-based memory pooling system using small pools (8-16 sockets) and ML-based allocation prediction, reduces DRAM costs by 7% while keeping performance within 1-5% of same-NUMA-node VM allocations across 158 workloads. | Cloud-specific; modest 7% cost saving; requires accurate ML prediction models. | high | abstract-only | Establishes that small CXL pools meet cloud performance goals, informing memory-pooling economics for LLM serving. |
| ganjihal2026topology | theoretical | Disaggregated prefill/decode GPU pools; 70B-class models; analytical bandwidth models | KV cache transfer latency, aggregate bandwidth demand | Projected analysis: KV-cache transfer for a 70B model is 2.6 GB per request (exceeding 100 GB/s aggregate at production scale), bandwidth between GPUs varies 72x (900 GB/s NVLink, 50 GB/s InfiniBand, 12.5 GB/s TCP), and topology-aware transport with pipelined transfer hides 60-85% of latency, giving 3-18x transfer-latency reduction over uniform RDMA. | Author-flagged: no experimental validation possible (multi-node heterogeneous interconnects and CXL 3.0 hardware unavailable); analytical projections only. | moderate | full-text | Quantifies the NVLink/InfiniBand/TCP bandwidth hierarchy (72x spread) that disaggregated inference systems must exploit, critiquing uniform-RDMA designs like DistServe/Splitwise/Mooncake. |
| zhong2024distserv | systems paper | Popular LLMs and applications on GPU serving clusters (A100-class hardware) | Served request rate within TTFT/TPOT SLO constraints | DistServe, which disaggregates prefill and decoding and places phases according to cluster bandwidth, serves 7.4x more requests or meets 12.6x tighter SLOs than state-of-the-art systems while staying within latency constraints for >90% of requests. | Requires bandwidth-aware placement; gains depend on workload and latency requirements. | high | full-text | Canonical prefill/decode disaggregation paper showing placement must account for interconnect bandwidth to minimize disaggregation communication. |
| patel2023splitwis | systems paper | A100/H100 GPU clusters; LLM inference characterization and cluster design (simulator-based) | Throughput, cost, power, KV state-transfer time | Splitwise phase-splitting clusters achieve 1.4x higher throughput at 20% lower cost, or alternatively 2.35x more throughput under the same cost and power budgets, with KV state transferred over fast back-plane interconnects (e.g., NVLink 50/100 Gbps, InfiniBand 200/400 GBps on A100/H100). | Cluster designs rely on a simulator; state-transfer overhead grows with model size and disaggregation distance. | high | full-text | Quantifies throughput/cost gains of phase splitting and the role of fast interconnects (NVLink/InfiniBand) in KV state transfer between phases. |
| qin2025mooncake | field study | Kimi LLM serving platform, thousands of nodes, NVIDIA A800/H800 clusters, real traces | Effective request capacity under SLOs, tokens processed per day | Mooncake, a KVCache-centric disaggregated serving platform processing over 100 billion tokens daily across thousands of nodes, increases effective request capacity by 59-498% versus baseline methods under SLOs and handles 115% (A800) and 107% (H800) more requests than previous systems. | Production-specific; benefits concentrated in long-context scenarios. | high | abstract-only | Production proof that disaggregated KVCache using spare CPU/DRAM/SSD/NIC resources scales LLM serving, with data-movement implications across interconnects. |
| wu2024loongser | systems paper | LoongServe LLM serving system evaluated on GPU clusters with NVLink and InfiniBand interconnects; real-world workload datasets (ShareGPT et al.); 1M-token context requests on 7B-class models | max throughput, KV cache migration overhead, per-token latency | LoongServe improves maximum throughput by up to 3.85x over chunked prefill and 5.81x over prefill-decoding disaggregation, with a single 1M-input request needing 488GB of KV cache. | Elastic scaling incurs KV cache migration cost over high-bandwidth interconnects; scheduling must fit tens-of-ms iteration budgets; evaluations on specific GPU clusters may not generalize. | high | full-text | Shows elastic sequence parallelism reduces KV-transfer/communication overhead in long-context serving, quantifying the interconnect cost of KV migration across NVLink/InfiniBand. |
| liu2025serverle | systems paper | ServerlessPD serverless LLM inference system with RDMA-based remote fork across machines | cold-start latency, KV cache state transfer cost, autoscaling latency | RDMA-co-designed remote fork enables near-instant autoscaling and zero-copy state (KV cache) transfer for serverless prefill-decode disaggregation (specific speedups not reported in abstract). | Abstract-only; no quantitative results or hardware scale reported in the available record. | moderate | abstract-only | Demonstrates RDMA kernel-integrated remote fork as a mechanism for zero-copy KV state migration in disaggregated serverless LLM serving. |
| liu2025lmcache | systems paper | LMCache KV caching layer with vLLM and SGLang; enterprise deployments; Qwen3-32B on 2x H100; 500GB CPU DRAM tier | throughput, TTFT, KV cache transfer/offload bandwidth, prefix cache hit ratio | Combining LMCache with vLLM achieves up to 15x throughput improvement, and real-world usage shows remote KV fetching benefits prefill delay while context truncation halves prefix cache hit ratio. | Evaluation uses stretched real traces run within one hour; results depend on storage/network backends (NFS, WEKA, GPU-Direct Storage, Mooncake Store, NIXL, S3, InfiniStore, Valkey). | high | full-text | Provides an open-source KV cache layer spanning GPU/CPU/storage/network tiers, quantifying gains of cross-engine KV movement for PD disaggregation and prefix reuse. |
| yang2026beyond | systems paper | DOPS framework on heterogeneous NPU-PIM systems; LLM inference workloads | geometric-mean end-to-end latency speedup, weight layout overhead | Bifocal scheduler achieves 1.20x-2.23x geometric-mean speedups over prefill-decode (PD) disaggregation baselines and WLA adds a further 1.28x-1.33x via hardware-efficient weight layouts. | Targets edge-oriented NPU-PIM platforms rather than GPU clusters; results are largely from performance-model-driven scheduling with limited real-system scale. | high | full-text | Argues PD disaggregation alone is insufficient and models communication/placement costs (all-reduce, gather, scatter) for operator scheduling across heterogeneous interconnects. |
| wei2024parallel | systems paper | DeTransformer distributed in-situ Transformer inference on 4 edge devices with weak interconnect | distributed inference latency, communication volume, accuracy | DeTransformer reduces distributed inference latency by up to 2.81x versus the SOTA approach on 4 devices while maintaining accuracy and model size. | Abstract-only; edge scenario with weak interconnect, small scale (4 devices), and no absolute bandwidth numbers reported. | moderate | abstract-only | Shows block-parallel inference restructuring to cut communication frequency under weak interconnects, relevant to interconnect-constrained distributed inference. |
| li2021terapipe | systems paper | TeraPipe token-level pipeline parallelism; GPT-3 models (1B-175B) on AWS cluster of 48 p3.16xlarge instances | per-iteration training latency, TFLOPs/GPU, speedup vs state-of-the-art model parallelism | TeraPipe speeds up GPT-3 175B training by 5.0x on 48 p3.16xlarge instances (1.40x for GPT3-13B, 1.88x/1.56x for GPT3-44B configurations). | Speedup vanishes when optimal slicing only uses the batch dimension; requires Transformer autoregressive structure; depends on cluster network bandwidth. | high | full-text | Exploits token-dimension pipelining to cut communication stalls in model-parallel LLM training; contrasts with allreduce-based Megatron-LM synchronization. |
| li2023sequence | systems paper | not reported (abstract record contains only citation metadata) | not reported | Proposes sequence parallelism for long-sequence training from a system perspective; no quantitative results available in the record. | Only citation metadata available; no abstract text, numbers, or methodology accessible locally. | low | abstract-only | Positional reference for sequence parallelism (SP) as an interconnect-relevant alternative to tensor parallelism for long-sequence LLM training. |
| brandon2023striped | systems paper | Striped Attention vs Ring Attention; A100 GPUs and TPUv4s; 1B/3B/7B causal transformers at 256k-786k sequence lengths | end-to-end training throughput, speedup over Ring Attention | Striped Attention achieves up to 1.45x end-to-end throughput over Ring Attention at 256k sequence length on A100/TPUv4 and 1.65x speedups on 16 TPUv4 chips at 786k. | Improvements specific to causal (triangular-mask) attention; more carefully optimized Ring/Striped implementations left to future work. | high | full-text | Improves load balance of ring-based context-parallel attention (KV communication overlapped with compute), relevant to long-context training communication patterns. |
| gu2024loongtra | systems paper | LoongTrain 2D-Attention; cluster of 8 GPU servers x 8 NVIDIA Ampere GPUs, NVLink intra-node, 4x 200Gb/s HDR InfiniBand NICs inter-node; sequence lengths to 1M | Model FLOPs Utilization (MFU), end-to-end training speed, SeqAlltoAll and P2P communication time | LoongTrain improves MFU by up to 2.88x over DeepSpeed-Ulysses and Megatron Context Parallelism, with Megatron-CP hitting out-of-memory at 1M sequence length. | Double-Ring-Attention gains depend on NIC count and device placement; GQA KV-replication trade-offs limit some configurations. | high | full-text | Combines head-parallel and context-parallel attention to use all inter-node NICs (HDR InfiniBand) efficiently, quantifying SeqAlltoAll vs P2P communication trade-offs. |
| ghadia2026untied | systems paper | UPipe (Untied Ulysses) headwise-chunked context parallelism; Llama3-8B on 8x H100 and Qwen3-32B on 16x H100; up to 8M-token contexts | intermediate tensor (QKV/all-to-all) activation memory, tokens/second/GPU throughput, max context length | UPipe cuts intermediate attention-layer tensor memory by up to 87.5% for 32B transformers and supports 5M-token context on a single 8xH100 node (25% beyond FPDT), scaling to 8M on 16 H100s at throughput comparable to baselines. | More head chunks increase runtime (memory-throughput trade-off); authors note limitations in the appendix scope section; multi-node configs use Ulysses+ring hybrids. | high | full-text | Shows headwise chunking to shrink all-to-all activation buffers in context-parallel training, addressing the memory side of long-context collective communication. |
| jiang2025addressi | systems paper | DCP dynamic context-parallel training framework; blockwise data/computation partitioning; micro-benchmarks plus end-to-end training | attention acceleration, end-to-end training speedup, communication overhead | DCP accelerates attention by 1.19x-2.45x under causal masks and 2.15x-3.77x under sparse attention, with 0.94x-1.16x (causal) and 1.00x-1.46x (sparse) end-to-end training speedups. | Abstract-only; gains depend on sequence-length variability and attention-pattern sparsity of the training data. | moderate | abstract-only | Motivates dynamic (rather than static) context-parallel partitioning to cut unnecessary communication for variable-length long-context training. |
| narayanan2021scale | systems paper | Megatron-LM tensor+pipeline+data parallelism composition; 1-trillion-parameter model on 3072 GPUs | training throughput (petaFLOP/s), per-GPU fraction of theoretical peak, scaling efficiency | Interleaved pipelining improves throughput by 10+% at comparable memory footprint, enabling 1T-parameter training at 502 petaFLOP/s on 3072 GPUs (52% of per-GPU theoretical peak). | Abstract-only; scaling results assume specific cluster interconnect; naive tensor/pipeline composition has scaling issues at thousands of GPUs. | moderate | abstract-only | Canonical reference on composing parallelisms to reach thousands of GPUs, setting the communication-efficiency baseline that interconnect-aware schedules target. |
| wang2020whale | systems paper | Whale framework on production Alibaba cluster of 512 GPUs (V100/P100 heterogeneous); M6 multimodal model with >10T parameters | training throughput, scaling efficiency, speedup from hardware-aware balancing | Whale trains the >10T-parameter M6 model on 512 GPUs with 95% scaling efficiency from 8 to 32 GPUs and 1.2x-1.4x speedups from hardware-aware load balancing (e.g., NVLink-aware AllReduce placement). | Speedups modest (1.2x-1.4x) for load balancing; evaluation largely on internal Alibaba workloads. | high | full-text | Demonstrates graph-level placement of collectives (e.g., NVLink-based AllReduce within nodes) as a lever for efficient giant-model training on heterogeneous clusters. |
| sensi2024explorin | benchmark | Three exascale supercomputers - Alps, Leonardo, LUMI - intra-node and inter-node interconnects on up to 4,096 GPUs | intra-node and inter-node interconnect bandwidth, latency, achieved vs peak throughput | Characterization shows untapped interconnect bandwidth and optimization opportunities across network and software layers (absolute bandwidth figures not reported in abstract). | Abstract-only; no numeric results in the record; findings tied to specific hardware generations and software stacks. | moderate | abstract-only | Provides comparative interconnect characterization (multi-Tbps intra-node networks) for GPU supercomputers, informing expectations for large-scale LLM training fabrics. |
| li2021scale | systems paper | 1-bit LAMB compressed communication; BERT-Large pre-training at batch sizes 8K-64K on up to 256 GPUs; Ethernet clusters with V100s | communication volume, end-to-end training time, convergence speed, allreduce share of step time | 1-bit LAMB with NCCL backend achieves up to 4.6x communication-volume reduction and up to 2.8x end-to-end speedup with identical convergence, where allreduce can consume up to 91% (and 52% on another cluster) of per-step time. | Compression trades some accuracy risk; extra memory for variance/momentum copies; benefits shrink on high-bandwidth clusters. | high | full-text | Quantifies allreduce-dominated communication on commodity TCP/Ethernet clusters and shows gradient compression as an interconnect-bandwidth mitigation for LLM-scale training. |
| li2024lossless | systems paper | GComp near-lossless gradient compression for data-parallel DNN training; multi-node GPU clusters | communication volume reduction, training speedup, accuracy preservation | GComp decreases synchronization communication volume by up to 67.1% and improves training speed by up to 1.9x with virtually no accuracy loss. | Abstract-only; effectiveness depends on gradient statistical characteristics and model/workload; compression ratio varies across models. | moderate | abstract-only | Near-lossless gradient compression as a way to relieve interconnect bandwidth pressure in data-parallel LLM-scale training. |
| li2024driven | systems paper | FreeINA host-driven in-network aggregation; P4-programmable switch + commercial RNICs on a 100Gbps testbed | single-job training speedup, aggregation throughput, switch memory efficiency | FreeINA improves single-job training speedup by 1.20x over ATP and aggregation throughput by 2.65x in multi-job scenarios while keeping RDMA RC compatibility. | Requires programmable switches; evaluation on 100Gbps testbed, not production-scale fabrics; multi-tenant assumptions may not hold in all settings. | moderate | abstract-only | Addresses the collision between in-network aggregation (INA) and RDMA reliable connections for DNN training, relevant to RDMA/RoCE-based training clusters. |
| segal2022constrai | theoretical | Algorithmic study of constrained in-network computing for distributed workloads in datacenter topologies (tree-like networks) | network congestion (max loaded link), reduction in congestion | An optimal algorithm for tree-like topologies reduces congestion by up to 13x, with even a small fraction of in-network-aggregation-capable devices giving significant gains. | Theoretical bounds for tree-like topologies only; practical switch constraints and multi-tenant dynamics simplified. | moderate | abstract-only | Formal foundation for in-network aggregation (e.g., SHARP-style) deployment, underpinning congestion reduction for distributed ML collectives. |
| zhu2024meets | survey | Survey of In-Network Computing (INC)-based distributed machine learning systems on programmable switches | qualitative performance improvement analysis, taxonomy of INC-based DML systems | Overview paper: INC accelerates DML training via switch programmability/compute/storage; no quantitative results reported. | Survey without experimental numbers; coverage limited to INC-based DML systems and research directions. | moderate | abstract-only | Survey framing INC (programmable-switch aggregation) as an accelerator for distributed ML, contextualizing switch-offload approaches for collectives. |
| wang2023blink | systems paper | Blink collective library on DGX-1P/DGX-1V/DGX-2 (NVLink, NVSwitch, PCIe); GPU counts 3-8+; data sizes 1MB-1000MB; cluster traces of 40,000 multi-GPU jobs | model synchronization time, broadcast/AllReduce throughput and latency, end-to-end training time | Blink achieves up to 8x faster model synchronization than NCCL and up to 40% lower end-to-end training time, with ~18GB/s throughput on datasets >10MB and up to 3.32x lower latency / 3.5x higher throughput on DGX-2 NVSwitch AllReduce. | Focused on image-classification-scale training; topology/heterogeneity assumptions (PCIe vs NVLink bandwidth) baked into tree packing; later NCCL versions close some gaps. | high | full-text | Spanning-tree packing over heterogeneous NVLink/PCIe/NIC topologies as an alternative to NCCL ring collectives, directly relevant to collective design for training. |
| ryohei2022mission | case study | Apollo optical circuit switching (OCS) production deployment in Google datacenters; 3D MEMS-based OCS with circulators and WDM transceivers over four interconnect generations | network bandwidth, scale, cost, OCS switching time, port count | World's first large-scale production OCS deployment uses circulators to double OCS radix and co-designed WDM transceivers to meet datacenter bandwidth/scale/cost targets (specific Gbps numbers not reported in available text). | Available record is the arXiv abstract page only (no full body); switching-time and performance details not extracted. | moderate | abstract-only | Production evidence for optical circuit switching in datacenter networks, a candidate technology for future AI-cluster fabric reconfiguration. |
| yan2023demystif | benchmark | First true CXL-ready systems: 4th-gen Intel Xeon (SPR) with three CXL memory devices from different vendors; microbenchmarks plus DLRM and other bandwidth-intensive workloads | CXL memory latency and bandwidth vs emulated CXL and DDR, application throughput, page-allocation ratio | Captión, a CXL-memory-aware dynamic page allocation policy, improves memory-bandwidth-intensive application performance by up to 24% versus default NUMA policies, and true CXL devices differ notably from DDR-emulated CXL. | Single-vendor CPU platform; results specific to SPR and the three tested CXL devices; CXL latency still higher than local DDR. | high | full-text | Empirical latency/bandwidth characterization of CXL memory expansion (and CXL vs RDMA remote access), informing memory-disaggregation options for LLM inference. |
| das2023introduc | survey | Survey of CXL 1.0/2.0/3.0 standards, implementations, and datacenter impact | qualitative: bandwidth scaling with PCIe, latency vs PCIe, ecosystem adoption | CXL provides coherency/memory semantics with bandwidth scaling with PCIe at significantly lower latency than PCIe, adopted by all major CPU/device vendors (no quantitative figures in record). | Available record is the arXiv abstract page only; survey-level without benchmark numbers. | moderate | abstract-only | Reference survey for CXL as an interconnect (memory expansion, pooling, peer-to-peer), a growing alternative/complement to RDMA for disaggregated memory. |
| angelos2024streamli | case study | ZeroPoint compressed CXL tiered-memory expander aligned with OCP Hyperscale specification; QEMU+FPGA proof of concept | compression ratio, cache-line access latency, TCO reduction | Hardware-accelerated lossless compressed CXL memory tier offers 2-3x compression in nanoseconds and 20-25% TCO reduction, meeting OCP targets of 250ns compressed-block access and <1us tail latency. | Industry position paper with emulation-based PoC; production silicon claims (mid-2024) not independently verified; CXL pool economics contested. | moderate | full-text | Shows compressed CXL tiers to expand effective memory capacity at hyperscale, addressing memory-capacity constraints in AI inference without extra network traffic. |
| saurabh2024symphony | systems paper | Symphony on 2 nodes x 4 NVIDIA A100 80GB HBM, 256GB DRAM, 4TB SSD per node, 100Gbps Ethernet inter-node; multi-turn chatbot and MetaGPT agent workloads | requests served per second, latency, load imbalance, redundant-token fraction | Symphony handles over 8x the requests of state-of-the-art baselines at similar latency, leveraging advisory requests issued on average 5.8s early (MetaGPT) and avoiding >99% redundant recomputed tokens. | Relies on advisory requests/hints from multi-turn workloads; hierarchical KV migration to host/disk adds complexity; benefits shrink without advisory info. | high | full-text | KV-cache migration off the critical path as an alternative to stateful offloading, illustrating state-transfer overheads across cluster interconnects (100Gbps Ethernet). |
| wanqian2026weight | systems paper | DWDP on NVIDIA GB200 NVL72 with DeepSeek-R1 (NVFP4 MoE); disaggregated serving, 8K input / 1K output sequences | output tokens/sec/GPU, TTFT, TPS/user, NVLink peer-to-peer prefetch overhead | DWDP improves end-to-end output TPS/GPU by 8.8% at 20-100 TPS/user (1.01-1.10x TPS/GPU and 1.07-1.16x TTFT across token-budget settings) by removing collective synchronization and prefetching MoE weights over NVLink. | Requires high-bandwidth all-to-all connectivity (NVL72-class); gains not monotonic in input length; copy-engine contention with compute kernels must be managed. | high | full-text | Replaces layer-wise all-to-all synchronization with async NVLink weight prefetch in MoE inference, quantifying interconnect-centric scheduling on NVL72. |
| k2026provisio | field study | Meta 150MW AI datacenter with 83K GB200 GPUs; end-to-end power management across planning, deployment, and runtime phases | cluster throughput, performance-per-watt, power limits, per-GPU RDMA bandwidth | The cluster doubles scale-out RDMA bandwidth per GPU vs the reference design via two 400Gbps CX7 NICs per Grace CPU, and a ~1000W GPU power limit (vs 1200W) improves total cluster throughput by ~9% (GB200 delivering 2.4x per-GPU performance of H100 at 960W). | Single-operator (Meta) production experience; power-centric framing; numbers tied to GB200-era hardware and specific workloads. | high | full-text | Production-scale datapoint on AI-cluster networking (2x400Gbps RDMA per GPU) and power/throughput trade-offs for 100MW-class LLM training clusters. |
| aashaka2023taccl | systems paper | TACCL collective synthesis for DGX-2 and NDv2 topologies (PCIe Gen3, NVLink, InfiniBand NICs); Allgather/AlltoAll/AllReduce; Transformer-XL and BERT training | collective time vs NCCL, end-to-end training speedup, synthesis time | TACCL-synthesized algorithms outperform NCCL by up to 6.7x (AllReduce up to 28% better for <=1MB buffers on NDv2) and speed up end-to-end Transformer-XL/BERT training by 11%-2.3x. | Synthesis can be slow (up to ~1800s for some sketches); requires designer-provided communication sketches; topology-specific results. | high | full-text | Automated synthesis of topology-aware collective algorithms (respecting PCIe/NVLink/IB link constraints), a key technique for optimizing all-to-all/allreduce in LLM training. |
| hao2023attentio | systems paper | Ring Attention on 8x A100 NVLink, 32x A100 InfiniBand, and TPUv3-512; 3B/7B/13B models; up to millions of tokens context | max context length, model FLOPs utilization (MFU), throughput, communication-computation overlap condition | Ring Attention trains/infers sequences up to device-count times longer (e.g., 256x on TPUv3-512 for 13B) by overlapping KV-block communication with attention compute, requiring block size >= FLOPs/bandwidth ratio (F/B). | Overlap requires sufficient bandwidth (e.g., with 186GB/s and 196 TFLOPS an MFU of ~40% is the break-even); ppermute-based ring communication only between adjacent hosts. | high | full-text | Foundational ring-attention result showing KV communication fully hidden behind compute; defines the bandwidth-to-FLOPS ratio governing long-context scaling. |
| amy2024context | systems paper | Context-parallel prefill on up to 128 H100 GPUs across 16 nodes (Grand Teton: 8x H100 NVLink per host, 96GB HBM2e, 2.4TB/s); inter-node RDMA and TCP; Llama3 405B | prefill latency (TTFT), parallelization efficiency, FLOPS utilization, KV cache capacity | Achieves 1M-token prefill of Llama3 405B in 77s (93% parallelization efficiency, 63% FLOPS utilization) and 128K prefill in 3.8s, with near-linear scaling on both RDMA- and TCP-interconnected hosts. | Power-limited H100s (500W, 2.4TB/s HBM) reduce peak FLOPS vs stock H100; All2All latency not in base heuristics (refined in appendix); decode-phase gains smaller. | high | full-text | Shows ring-based context parallelism scales near-linearly to 128 GPUs even on medium/low-bandwidth interconnects, key evidence for long-context inference over commodity fabrics. |
| mohammad2020megatron | systems paper | Megatron-LM tensor parallelism; up to 8.3B-parameter transformer (GPT-2-like) on 512 GPUs (DGX-2H class) | sustained petaFLOPs, scaling efficiency, FLOPS fraction of peak, communication ops per layer | Megatron-LM sustains 15.1 PetaFLOPs across 512 GPUs (76% scaling efficiency vs a 39-TFLOP single-GPU baseline at 30% of peak) using only 2 all-reduces forward and 2 backward per transformer layer. | Intra-layer parallelism only; requires high-bandwidth intra-node interconnects for the all-reduce-heavy scheme; data-parallel groups add inter-node traffic. | high | full-text | Seminal tensor-parallel design minimizing allreduce count per layer; the communication baseline against which later interconnect-aware LLM training systems are measured. |
| braun2020analyz | benchmark | InfiniBand EDR 100Gbps vs HDR 200Gbps over PCIe Gen3 with HPC, Cloud, and Deep Learning workloads | application-level performance across interconnect generations (bandwidth, latency, DL training throughput) | The latest InfiniBand HDR interconnect delivers the best performance across all three computing domains (specific speedup numbers not reported in abstract). | Abstract-only; PCIe Gen3 host interface may bottleneck newer NICs; results tied to 2020-era hardware and middleware. | moderate | abstract-only | Quantifies end-user benefit of moving from InfiniBand EDR to HDR generations, grounding expectations for RDMA fabric upgrades in DL training. |

## References

1. Gangidi, Adithya et al. (2024). *RDMA over Ethernet for Distributed Training at Meta Scale*. arXiv preprint. Abstract only. Production evidence that RoCE/Ethernet is viable as the interconnect fabric for large-scale AI training. [doi:10.1145/3651890.3672233](https://doi.org/10.1145/3651890.3672233)
2. Batlle Casellas, Alex et al. (2025). *Scaling LLM Training Using RDMA over Converged Ethernet*. Proceedings of the SC '25 Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis. Abstract only. Head-to-head benchmark of RoCE vs InfiniBand for LLM training, supporting cost-effective Ethernet-based interconnects. [doi:10.1145/3731599.3767453](https://doi.org/10.1145/3731599.3767453)
3. Hoefler, Torsten et al. (2023). *Datacenter Ethernet and RDMA: Issues at Hyperscale*. arXiv preprint. Full text read. Foundational critique of RoCE/PFC (headroom buffers, victim flows, go-back-N, ECN) that motivates Ultra Ethernet for AI datacenters. [doi:10.48550/arxiv.2302.03337](https://doi.org/10.48550/arxiv.2302.03337)
4. Khan, Tarannum et al. (2022). *Impact of RoCE Congestion Control Policies on Distributed Training of DNNs*. arXiv preprint. Full text read. Shows general-datacenter congestion control does not transfer to collective-based (All-Reduce/All-To-All) training networks, guiding interconnect CC design. [doi:10.48550/arxiv.2207.10898](https://doi.org/10.48550/arxiv.2207.10898)
5. He, Xin et al. (2025). *Rate-adaptive RDMA congestion control for AI clusters*. Journal of Cloud Computing. Abstract only. RDMA congestion control optimization for latency-sensitive AI applications on lossless fabrics. [doi:10.1186/s13677-025-00830-0](https://doi.org/10.1186/s13677-025-00830-0)
6. Li, Wenxue et al. (2025). *Revisiting RDMA Reliability for Lossy Fabrics*. arXiv preprint. Abstract only. Enables efficient RDMA over lossy (non-PFC) Ethernet fabrics, relevant to RoCE scalability for LLM training. [doi:10.1145/3718958.3750480](https://doi.org/10.1145/3718958.3750480)
7. Khalilov, Mikhail et al. (2025). *SDR-RDMA: Software-Defined Reliability Architecture for Planetary Scale RDMA Communication*. arXiv preprint. Full text read. Software-defined reliability stack for planetary-scale RDMA, key for multi-datacenter LLM training collectives. [doi:10.48550/arxiv.2505.05366](https://doi.org/10.48550/arxiv.2505.05366)
8. Lin, Shengkai et al. (2025). *SHIFT: Exploring the Boundary of RDMA Network Fault Tolerance*. arXiv preprint. Full text read. Extends RDMA fault tolerance to cross-NIC level and quantifies network failures' share in large-scale LLM training. [doi:10.48550/arxiv.2512.11094](https://doi.org/10.48550/arxiv.2512.11094)
9. Hua, Xiaoxiang et al. (2026). *Communication-Semantic-Aware RDMA Loss Recovery for QP-scalable Hyperscale AI Training*. arXiv preprint. Full text read. Addresses QP scalability and tail latency of RDMA collectives (All-Reduce/All-to-All) for trillion-parameter training. [doi:10.48550/arxiv.2606.20582](https://doi.org/10.48550/arxiv.2606.20582)
10. Yang, Zhiyi et al. (2025). *DCTS-RDMA: Adaptive FEC via Dynamic Coding for Efficient RDMA over Lossy Networks*. Lecture Notes in Computer Science. Abstract only. FEC-based alternative to lossless fabrics for RDMA, relevant to RoCE reliability design. [doi:10.1007/978-3-032-10459-5_19](https://doi.org/10.1007/978-3-032-10459-5_19)
11. Snyder, John et al. (2023). *RDMA Congestion Control: It Is Only for the Compliant*. IEEE Micro. Abstract only. Security/isolation angle on RDMA congestion control for shared AI clouds. [doi:10.1109/mm.2022.3208746](https://doi.org/10.1109/mm.2022.3208746)
12. Wan, Zirui et al. (2025). *RHCC: Revisiting Intra-Host Congestion Control in RDMA Networks*. IEEE Transactions on Networking. Abstract only. Highlights intra-host congestion as an emerging bottleneck as RNIC speeds outpace host interconnects. [doi:10.1109/ton.2024.3524247](https://doi.org/10.1109/ton.2024.3524247)
13. Shan, Danfeng et al. (2026). *Efficient Headroom Allocation With Two-Level Flow Control for Lossless Datacenter Networks*. IEEE Transactions on Networking. Abstract only. Headroom buffer inefficiency in lossless Ethernet, a core cost of PFC-based RoCE deployments. [doi:10.1109/ton.2025.3596437](https://doi.org/10.1109/ton.2025.3596437)
14. Liu, Jingling et al. (2026). *FAR: Fast and Accurate Rate Control for Lossless Datacenter Networks*. IEEE Transactions on Networking. Abstract only. Combines flow pausing and congestion control for low-latency datacenter (incl. RDMA) networks. [doi:10.1109/ton.2026.3655983](https://doi.org/10.1109/ton.2026.3655983)
15. Planeta, Maksym et al. (2025). *CoRD: Converged RDMA Dataplane*. 2025 IEEE International Parallel and Distributed Processing Symposium (IPDPS). Abstract only. Challenges the kernel-bypass assumption of RDMA stacks, relevant to cloud-hosted RDMA AI training. [doi:10.1109/ipdps64566.2025.00099](https://doi.org/10.1109/ipdps64566.2025.00099)
16. Li, Wenxue et al. (2023). *Gleam: An RDMA-accelerated Multicast Protocol for Datacenter Networks*. arXiv preprint. Full text read. Brings reliable multicast to RDMA RC transport, a building block for group communication in training frameworks. [doi:10.48550/arxiv.2307.14074](https://doi.org/10.48550/arxiv.2307.14074)
17. Tian, F. et al. (2021). *Accelerating Distributed Deep Learning using Multi-Path RDMA in Data Center Networks*. arXiv preprint. Abstract only. Software multi-path load balancing for RDMA, addressing the single-path limitation of RNIC hardware. [doi:10.1145/3482898.3483363](https://doi.org/10.1145/3482898.3483363)
18. Luo, Changyun et al. (2024). *FlowStar: Fast Convergence Per-Flow State Accurate Congestion Control for InfiniBand*. IEEE/ACM Transactions on Networking. Abstract only. InfiniBand congestion control addressing victim flows and slow-convergence congestion spreading. [doi:10.1109/tnet.2024.3363658](https://doi.org/10.1109/tnet.2024.3363658)
19. Cascajo, Alberto et al. (2023). *Monitoring InfiniBand Networks to React Efficiently to Congestion*. IEEE Micro. Abstract only. Operational tuning of InfiniBand congestion control parameters in real HPC clusters. [doi:10.1109/mm.2023.3241840](https://doi.org/10.1109/mm.2023.3241840)
20. Maglione-Mathey, German et al. (2020). *Path2SL: Leveraging InfiniBand Resources to Reduce Head-of-Line Blocking in Fat Trees*. IEEE Micro. Abstract only. InfiniBand queuing/Virtual Lane mechanisms for congestion mitigation in HPC interconnects. [doi:10.1109/mm.2019.2949280](https://doi.org/10.1109/mm.2019.2949280)
21. Katebzadeh, M. R. Siavash et al. (2020). *Evaluation of an InfiniBand Switch: Choose Latency or Bandwidth, but Not Both*. arXiv preprint. Abstract only. Measurement methodology (RPerf) and evidence on InfiniBand switch behavior under mixed workloads. [doi:10.1109/ispass48437.2020.00033](https://doi.org/10.1109/ispass48437.2020.00033)
22. Luo, Changyun et al. (2024). *Alarm: An Adaptive Routing Algorithm Based on One-Way Delay for Infiniband*. IEEE Transactions on Network Science and Engineering. Abstract only. Adaptive routing to balance traffic in InfiniBand-dominated HPC/AI clusters. [doi:10.1109/tnse.2024.3382295](https://doi.org/10.1109/tnse.2024.3382295)
23. Wei, Ying et al. (2023). *9.3 NVLink-C2C: A Coherent Off Package Chip-to-Chip Interconnect with 40Gbps/pin Single-ended Signaling*. arXiv preprint. Abstract only. Documents NVLink-C2C as the high-bandwidth CPU-GPU interconnect enabling unified memory for AI/HPC. [doi:10.1109/isscc42615.2023.10067395](https://doi.org/10.1109/isscc42615.2023.10067395)
24. Ishii, Alexander T. & Wells, Ryan S. (2022). *The Nvlink-Network Switch: Nvidia’s Switch Chip for High Communication-Bandwidth Superpods*. arXiv preprint. Abstract only. Historical context source for NVLink; minimal extractable evidence. [doi:10.1109/hcs55958.2022.9895480](https://doi.org/10.1109/hcs55958.2022.9895480)
25. Li, Ang et al. (2020). *Evaluating Modern GPU Interconnect: PCIe, NVLink, NV-SLI, NVSwitch and GPUDirect*. IEEE Transactions on Parallel and Distributed Systems. Abstract only. Empirical characterization of intra-node GPU interconnects (PCIe/NVLink/NVSwitch) foundational to multi-GPU training. [doi:10.1109/tpds.2019.2928289](https://doi.org/10.1109/tpds.2019.2928289)
26. Shen, Ao et al. (2025). *FlexLink: Boosting your NVLink Bandwidth by 27% without accuracy concern*. arXiv preprint. Full text read. Directly addresses intra-node interconnect utilization (NVLink+PCIe+RDMA aggregation) for LLM training/inference collectives. [doi:10.48550/arxiv.2510.15882](https://doi.org/10.48550/arxiv.2510.15882)
27. Werner, Felix et al. (2025). *Towards Memory Disaggregation via NVLink C2C: Benchmarking CPU-Requested GPU Memory Access*. arXiv preprint. Abstract only. Quantifies NVLink-C2C viability for memory expansion, informing memory-disaggregation interconnect choices. [doi:10.1145/3723851.3723853](https://doi.org/10.1145/3723851.3723853)
28. Vemuri, Sai Krishna et al. (2025). *Communication-Centric UALink and NVLink Comparison for Large Scale AI Training*. 2025 IEEE 32nd International Conference on High Performance Computing, Data and Analytics Workshop (HiPCW). Abstract only. Compares open UALink vs proprietary NVLink for AI-training collectives, informing interconnect standardization. [doi:10.1109/hipcw66559.2025.00020](https://doi.org/10.1109/hipcw66559.2025.00020)
29. Choi, Yea Rem & Stegailov, Vladimir (2022). *Multi-GPU GEMM Algorithm Performance Analysis for Nvidia and AMD GPUs Connected by NVLink and PCIe*. Communications in computer and information science. Abstract only. Intra-node NVLink/NVSwitch performance analysis for multi-GPU compute kernels. [doi:10.1007/978-3-031-24145-1_23](https://doi.org/10.1007/978-3-031-24145-1_23)
30. Luo, Shutian et al. (2026). *C2CServe: Leveraging NVLink-C2C for Elastic Serverless LLM Serving on MIG*. arXiv preprint. Full text read. Shows NVLink-C2C bandwidth reshaping LLM inference/serving architecture, an interconnect-centric serving system. [doi:10.48550/arxiv.2605.19481](https://doi.org/10.48550/arxiv.2605.19481)
31. Zhang, Yicheng et al. (2025). *NVBleed: Covert and Side-Channel Attacks on NVIDIA Multi-GPU Interconnect*. arXiv preprint. Full text read. Security analysis of the NVLink interconnect in multi-GPU and cloud AI systems, documenting leakage via contention and counters. [doi:10.48550/arxiv.2503.17847](https://doi.org/10.48550/arxiv.2503.17847)
32. Zhang, Yicheng et al. (2024). *Beyond the Bridge: Contention-Based Covert and Side Channel Attacks on Multi-GPU Interconnect*. arXiv preprint. Abstract only. Demonstrates NVLink congestion leakage as a side channel in multi-GPU systems. [doi:10.1109/seed61283.2024.00014](https://doi.org/10.1109/seed61283.2024.00014)
33. Singhania, Prajwal et al. (2025). *Understanding and Improving Communication Performance in Multi-node LLM Inference*. arXiv preprint. Full text read. Shows all-reduce/inter-node interconnect (Slingshot, InfiniBand) as the key bottleneck in multi-node LLM inference. [doi:10.48550/arxiv.2511.09557](https://doi.org/10.48550/arxiv.2511.09557)
34. Hoefler, Torsten et al. (2025). *Ultra Ethernet's Design Principles and Architectural Innovations*. arXiv preprint. Full text read. The Ethernet ecosystem's answer to InfiniBand for AI/HPC: defines the modernized high-performance Ethernet transport. [doi:10.48550/arxiv.2508.08906](https://doi.org/10.48550/arxiv.2508.08906)
35. Metz, Joachim (2024). *Empowering AI Workloads in Ultra Ethernet Consortium*. arXiv preprint. Abstract only. Motivation and consortium context for Ultra Ethernet as the AI interconnect standard. [doi:10.1109/sum60964.2024.10614558](https://doi.org/10.1109/sum60964.2024.10614558)
36. Arsid, Rajesh (2026). *UALink, Ultra Ethernet, and PCIe: Transforming Next-Generation HPC and AI Workloads*. Journal of Information Systems Engineering & Management. Abstract only. Positions UALink/Ultra Ethernet/CXL as the emerging hierarchical interconnect stack for scale-out AI, directly relevant to RDMA/Ethernet/RoCE and scale-up fabrics for LLM training clusters. [doi:10.52783/jisem.v11i2s.14455](https://doi.org/10.52783/jisem.v11i2s.14455)
37. Alladi, Vyshnavi & Sass, Ronald (2026). *Impact of Ultra Ethernet Transport on Co-Tenant Latency in Reconfigurable Data Center NICs: A Simulation Study*. arXiv preprint. Abstract only. Supports incremental adoption of Ultra Ethernet (RoCE-class) transport sharing NICs with legacy tenants in hyperscaler clouds for AI/HPC traffic. [doi:10.1109/fccm68464.2026.00080](https://doi.org/10.1109/fccm68464.2026.00080)
38. Hu, Zhiyi et al. (2025). *Demystifying NCCL: An In-depth Analysis of GPU Communication Protocols and Algorithms*. arXiv preprint. Full text read. Foundational reference demystifying NCCL internals (protocol variants, GPUDirect RDMA transports, ring/tree algorithms) that underpin large-scale LLM training interconnect stacks. [doi:10.48550/arxiv.2507.04786](https://doi.org/10.48550/arxiv.2507.04786)
39. Weingram, Adam et al. (2023). *xCCL: A Survey of Industry-Led Collective Communication Libraries for Deep Learning*. Journal of Computer Science and Technology. Abstract only. Maps the collective communication library landscape (NCCL/RCCL/MSCCL/ACCL/Gloo) relevant to interconnect-aware optimization for deep learning workloads. [doi:10.1007/s11390-023-2894-6](https://doi.org/10.1007/s11390-023-2894-6)
40. Cowan, Meghan et al. (2023). *MSCCLang: Microsoft Collective Communication Language*. arXiv preprint. Abstract only. Enables topology-aware custom collectives (e.g., for NVLink/InfiniBand topologies), an approach later built upon by synthesis-based schedulers such as TACCL and Canvas. [doi:10.1145/3575693.3575724](https://doi.org/10.1145/3575693.3575724)
41. Hei, Chenyang et al. (2025). *Canvas: Scalable Collective Communication Scheduling for Large-Scale GPU Clusters*. 2025 IEEE 33rd International Conference on Network Protocols (ICNP). Abstract only. Advances scalable, near-optimal topology-aware collective scheduling to overcome vendor-library bandwidth bottlenecks in large GPU clusters used for LLM training. [doi:10.1109/icnp65844.2025.11192367](https://doi.org/10.1109/icnp65844.2025.11192367)
42. Cao, Jiamin et al. (2024). *Crux: GPU-Efficient Communication Scheduling for Deep Learning Training*. Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication. Abstract only. Shows communication contention among co-located DLT jobs wastes GPU compute and that prioritizing high-intensity flows mitigates it in shared training clusters. [doi:10.1145/3651890.3672239](https://doi.org/10.1145/3651890.3672239)
43. Huang, Jiajun et al. (2023). *gZCCL: Compression-Accelerated Collective Communication Framework for GPU Clusters*. arXiv preprint. Full text read. Demonstrates accuracy-aware lossy compression inside GPU-aware collectives as a way to stretch limited inter-node bandwidth (100 Gbps-class) in large-scale training. [doi:10.48550/arxiv.2308.05199](https://doi.org/10.48550/arxiv.2308.05199)
44. Li, Baojia et al. (2024). *TCCL: Co-optimizing Collective Communication and Traffic Routing for GPU-centric Clusters*. arXiv preprint. Abstract only. Field evidence that GPU-centric networks (>10K GPUs) require co-design of collective communication and centralized routing to avoid congestion and jitter in LLM training. [doi:10.1145/3672198.3673799](https://doi.org/10.1145/3672198.3673799)
45. Liu, Yipeng et al. (2026). *UBEP: Re-architecting Expert Parallelism Communication Library for Production Superpods*. SIGCOMM 2026. Full text read. Quantifies how BSP serialization, synchronization overhead, and distance-agnostic scheduling waste high-bandwidth scale-up fabrics (UB/NVLink-class) in MoE all-to-all traffic. [doi:10.48550/arxiv.2607.06202](https://doi.org/10.48550/arxiv.2607.06202)
46. Goldman, Amos et al. (2026). *NCCL EP: Towards a Unified Expert Parallel Communication API for NCCL*. arXiv preprint. Full text read. Brings MoE dispatch/combine (fine-grained all-to-all over RDMA+NVLink) natively into NCCL, unifying device-initiated expert-parallel communication for training and inference. [doi:10.48550/arxiv.2603.13606](https://doi.org/10.48550/arxiv.2603.13606)
47. Zheng, Yusheng (2026). *NCCLbpf: Verified, Composable Policy Execution for GPU Collective Communication*. arXiv preprint. Full text read. Demonstrates verified, composable, hot-reloadable policy extension (eBPF-style) for NCCL's plugin interface, improving collective throughput without modifying NCCL source. [doi:10.48550/arxiv.2603.11438](https://doi.org/10.48550/arxiv.2603.11438)
48. Jia, Ziyang et al. (2026). *NIXT: A NCCL Inspector Exporter Tool for Observability of Collective Communication in Large Model Training*. arXiv preprint. Full text read. Provides production-scale observability into collective communication (bandwidth, stragglers, phase attribution) needed to diagnose interconnect bottlenecks in LLM pretraining. [doi:10.48550/arxiv.2608.01449](https://doi.org/10.48550/arxiv.2608.01449)
49. Shen, Siyuan et al. (2026). *Every Microsecond Matters: Achieving Near Speed-of-Light Latency in GPU Collectives*. arXiv preprint. Full text read. Shows latency, not just bandwidth, dominates small collectives on decode-heavy LLM inference critical paths, and quantifies microsecond-level gains achievable on scale-up interconnects. [doi:10.48550/arxiv.2607.16100](https://doi.org/10.48550/arxiv.2607.16100)
50. Singh, Siddharth et al. (2025). *The Big Send-off: Scalable and Performant Collectives for Deep Learning*. arXiv preprint. Full text read. Quantifies large headroom in vendor collective libraries (RCCL/NCCL) at supercomputer scale, informing interconnect-aware collective design for large-buffer DL workloads. [doi:10.48550/arxiv.2504.18658](https://doi.org/10.48550/arxiv.2504.18658)
51. Zhang, Mingjun et al. (2025). *An Efficient, Reliable and Observable Collective Communication Library in Large-scale GPU Training Clusters*. arXiv preprint. Full text read. Production evidence that SM competition, RNIC port failures, and transient RDMA anomalies waste GPU resources, and that SM-free P2P plus fault-tolerant QPs recover most of the loss. [doi:10.48550/arxiv.2510.00991](https://doi.org/10.48550/arxiv.2510.00991)
52. Zhang, Lin et al. (2023). *DeAR: Accelerating Distributed Deep Learning with Fine-Grained All-Reduce Pipelining*. arXiv preprint. Full text read. Shows fine-grained overlap of decoupled all-reduce with both backward and forward passes, with larger gains on low-bandwidth/high-latency Ethernet than on InfiniBand. [doi:10.48550/arxiv.2302.12445](https://doi.org/10.48550/arxiv.2302.12445)
53. Wang, Guanhua et al. (2023). *ZeRO++: Extremely Efficient Collective Communication for Giant Model Training*. arXiv (Cornell University). Full text read. Communication-volume reduction techniques that make ZeRO-3 viable on bandwidth-limited Ethernet clusters, a key regime for LLM training outside InfiniBand/NVLink datacenters. [doi:10.48550/arxiv.2306.10209](https://doi.org/10.48550/arxiv.2306.10209)
54. Park, Gyeongseo et al. (2026). *FCDP: Fully Cached Data Parallel for Communication-Avoiding Large-Scale Training*. arXiv.org. Abstract only. Reframes host memory as a fast caching layer to bypass slow inter-node interconnects, a practical trick for LLM training on non-NVLink/InfiniBand commodity fabrics. [doi:10.48550/arxiv.2602.06499](https://doi.org/10.48550/arxiv.2602.06499)
55. Xu, Guanbin et al. (2026). *Lagom: Unleashing the Power of Communication and Computation Overlapping for Distributed LLM Training*. arXiv preprint. Full text read. Shows communication tuning must also target computation-bottlenecked regimes, informing overlap strategies on both RDMA and NVLink fabrics in LLM training. [doi:10.48550/arxiv.2602.20656](https://doi.org/10.48550/arxiv.2602.20656)
56. Jia, Ziyang et al. (2024). *PCCL: Energy-Efficient LLM Training with Power-Aware Collective Communication*. 2024 IEEE 42nd International Conference on Computer Design (ICCD). Abstract only. Adds an energy dimension to interconnect/collective design: communication kernels dominate training time and offer DVFS-based power savings with minimal performance cost. [doi:10.1109/iccd63220.2024.00023](https://doi.org/10.1109/iccd63220.2024.00023)
57. Jin, Xin et al. (2024). *SDCC: software-defined collective communication for distributed training*. Science China Information Sciences. Abstract only. Software-defined, transport-agnostic collectives (including kernel-bypass/RDMA-class transports) as an alternative to tightly coupled vendor communication stacks. [doi:10.1007/s11432-023-3894-4](https://doi.org/10.1007/s11432-023-3894-4)
58. Li, Wenxue et al. (2024). *Understanding Communication Characteristics of Distributed Training*. arXiv preprint. Abstract only. Empirical basis for predicting communication overhead in distributed LLM training, informing when interconnect optimizations (RDMA, collectives) matter most. [doi:10.1145/3663408.3663409](https://doi.org/10.1145/3663408.3663409)
59. Samsi, Siddharth et al. (2020). *Benchmarking network fabrics for data distributed training of deep neural networks*. arXiv preprint. Abstract only. Benchmark evidence that for modest data-parallel workloads interconnect choice (Ethernet vs OmniPath) matters little, contrasting with today's bandwidth-hungry LLM training. [doi:10.1109/hpec43674.2020.9286232](https://doi.org/10.1109/hpec43674.2020.9286232)
60. Chen, Peiqing et al. (2026). *Don't Let a Few Network Failures Slow the Entire AllReduce*. arXiv preprint. Full text read. Quantifies and nearly eliminates the AllReduce penalty of NIC-failure rerouting (e.g., PXN-style failover) that otherwise throttles entire data-parallel training collectives. [doi:10.48550/arxiv.2606.01680](https://doi.org/10.48550/arxiv.2606.01680)
61. Wang, Wei et al. (2025). *Reliable and Resilient Collective Communication Library for LLM Training and Serving*. arXiv.org. Abstract only. Fault-tolerant collective library exploiting multi-NIC RDMA connection migration to prevent job-killing timeouts and checkpoint rollbacks in InfiniBand-based training clusters. [doi:10.48550/arxiv.2512.25059](https://doi.org/10.48550/arxiv.2512.25059)
62. Pan, Lichen et al. (2025). *Comprehensive Deadlock Prevention for GPU Collective Communication*. Proceedings of the Twentieth European Conference on Computer Systems. Abstract only. Catalogues correctness hazards (deadlocks) in collective communication stacks that practitioners must handle when orchestrating interconnect-heavy LLM training. [doi:10.1145/3689031.3717466](https://doi.org/10.1145/3689031.3717466)
63. Arzani, Behnaz et al. (2023). *Rethinking Machine Learning Collective Communication as a Multi-Commodity Flow Problem*. arXiv (Cornell University). Full text read. Recasts ML collective scheduling as a traffic-engineering-style flow problem, producing scalable, higher-quality schedules for AllToAll/AllGather/AllReduce on GPU fabrics. [doi:10.48550/arxiv.2305.13479](https://doi.org/10.48550/arxiv.2305.13479)
64. Hamidouche, Khaled et al. (2025). *GPU-Initiated Networking for NCCL*. arXiv preprint. Full text read. Documents the hardware/software path for GPU-initiated RDMA (GPUDirect Async / DOCA GPUNetIO), central to low-latency MoE all-to-all on InfiniBand/RoCE fabrics. [doi:10.48550/arxiv.2511.15076](https://doi.org/10.48550/arxiv.2511.15076)
65. Hwang, Chang Ho et al. (2022). *Tutel: Adaptive Mixture-of-Experts at Scale*. arXiv (Cornell University). Full text read. Adaptive parallelism and hierarchical all-to-all for MoE, addressing dynamic token-routing traffic patterns on A100 clusters relevant to LLM-scale expert parallelism. [doi:10.48550/arxiv.2206.03382](https://doi.org/10.48550/arxiv.2206.03382)
66. Rajbhandari, Samyam et al. (2022). *DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale*. arXiv (Cornell University). Full text read. Establishes MoE all-to-all communication as a first-class cost in large-model training/inference systems, motivating interconnect- and RDMA-aware expert-parallel design. [doi:10.48550/arxiv.2201.05596](https://doi.org/10.48550/arxiv.2201.05596)
67. DeepSeek-AI et al. (2024). *DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model*. arXiv (Cornell University). Full text read. Production MoE architecture whose device-limited routing and communication-balance losses directly shape all-to-all traffic on GPU interconnects during training and inference. [doi:10.48550/arxiv.2405.04434](https://doi.org/10.48550/arxiv.2405.04434)
68. Dai, Damai et al. (2024). *DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models*. arXiv preprint. Abstract only. Citation-only record of the DeepSeekMoE architecture paper; relevant as MoE routing/expert-specialization context for all-to-all communication in LLM training. [doi:10.18653/v1/2024.acl-long.70](https://doi.org/10.18653/v1/2024.acl-long.70)
69. DeepSeek-AI et al. (2024). *DeepSeek-V3 Technical Report*. arXiv (Cornell University). Full text read. Flagship evidence that computation-communication overlap plus IB/NVLink-optimized all-to-all kernels make 671B-scale MoE training communication-near-free and cost-effective. [doi:10.48550/arxiv.2412.19437](https://doi.org/10.48550/arxiv.2412.19437)
70. Nie, Xiaonan et al. (2024). *LSH-MoE: Communication-efficient MoE Training via Locality-Sensitive Hashing*. arXiv (Cornell University). Full text read. Quantifies all-to-all as the dominant cost in MoE training (45%+ of time on 200 Gb/s fabrics) and shows token-similarity compression as a communication-volume mitigation. [doi:10.48550/arxiv.2411.08446](https://doi.org/10.48550/arxiv.2411.08446)
71. Zhang, Zhonghao et al. (2026). *HALO: Hiding All-to-all via Locality-aware Overlap for Efficient MoE Training*. arXiv preprint. Abstract only. Shows all-to-all communication in expert-parallel MoE training can be hidden via locality-aware overlap and shadow experts, a key technique for interconnect-bound LLM training. [doi:10.2139/ssrn.6913743](https://doi.org/10.2139/ssrn.6913743)
72. Qi, Shuyao et al. (2026). *FEPLB: Exploiting Copy Engines for Nearly Free MoE Load Balancing in Distributed Training*. arXiv preprint. Full text read. Exploits the NVLink Copy Engine as a near-free intra-node communication channel parallel to compute kernels, informing NVLink usage for MoE load balancing in LLM training. [doi:10.48550/arxiv.2604.19654](https://doi.org/10.48550/arxiv.2604.19654)
73. Qin, Huihuang et al. (2026). *Reducing Cross-Pod Communication Overhead for MoE Model Training With Hybrid Parallelism in Multi-Tenant Clusters*. IEEE Transactions on Parallel and Distributed Systems. Abstract only. Addresses cross-Pod placement to mitigate intra-/inter-Pod bandwidth and latency discrepancies for MoE training, relevant to interconnect-aware job placement. [doi:10.1109/tpds.2026.3668417](https://doi.org/10.1109/tpds.2026.3668417)
74. Zhang, Qijun et al. (2026). *Accelerating MoE with Dynamic In-Switch Computing on Multi-GPUs*. arXiv preprint. Full text read. Extends NVLink SHARP in-switch reduction to irregular dynamic MoE traffic, showing in-switch computing can cut redundant dispatch/combine transfers in LLM training. [doi:10.48550/arxiv.2605.05607](https://doi.org/10.48550/arxiv.2605.05607)
75. Zhu, Ruidong et al. (2025). *MegaScale-Infer: Efficient Mixture-of-Experts Model Serving with Disaggregated Expert Parallelism*. arXiv preprint. Abstract only. Partial abstract frames memory-bound MoE inference efficiency, relevant to memory/interconnect pressure in LLM serving. [doi:10.1145/3718958.3750506](https://doi.org/10.1145/3718958.3750506)
76. Cai, Weilin et al. (2024). *Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Experts*. arXiv (Cornell University). Full text read. Quantifies how NVLink bandwidth (A30 vs A800) changes All-to-All overhead and shows shortcut-connected MoE decouples communication from computation for overlap. [doi:10.48550/arxiv.2404.05019](https://doi.org/10.48550/arxiv.2404.05019)
77. Wang, Weiyang et al. (2022). *TopoOpt: Co-optimizing Network Topology and Parallelization Strategy for Distributed Training Jobs*. arXiv preprint. Full text read. Pioneers co-optimizing network topology with AllReduce traffic mutability for DNN training, foundational for topology-aware interconnect design (RDMA/OCS). [doi:10.48550/arxiv.2202.00433](https://doi.org/10.48550/arxiv.2202.00433)
78. Namkoong, Seok et al. (2025). *SortingHat: System Topology-aware Scheduling of Deep Neural Network Models on Multi-GPU Systems*. Proceedings of the 39th ACM International Conference on Supercomputing. Abstract only. Shows multi-GPU LLM inference gains depend on co-scheduling data-transfer parallelism with system topology, relevant to interconnect-aware scheduling. [doi:10.1145/3721145.3728490](https://doi.org/10.1145/3721145.3728490)
79. Chang, Zihan et al. (2026). *SpiderFlow: Efficient Topology-Aware Scheduling for LLM Training Across Decentralized GPU Clusters*. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Abstract only. Citation-only record (SpiderFL, ACL 2026); likely federated-learning communication/scheduling work but no extractable interconnect evidence. [doi:10.18653/v1/2026.acl-long.619](https://doi.org/10.18653/v1/2026.acl-long.619)
80. Ye, Zhisheng et al. (2023). *Deep Learning Workload Scheduling in GPU Datacenters: A Survey*. ACM Computing Surveys. Abstract only. Provides a taxonomy of GPU-datacenter scheduling including communication- and resource-aware methods, context for interconnect-aware training scheduling. [doi:10.1145/3638757](https://doi.org/10.1145/3638757)
81. Yu, Menglu et al. (2022). *On scheduling ring-all-reduce learning jobs in multi-tenant GPU clusters with communication contention*. arXiv preprint. Abstract only. Models communication contention between co-located all-reduce training jobs, relevant to RDMA/collective scheduling in GPU clusters. [doi:10.1145/3492866.3549716](https://doi.org/10.1145/3492866.3549716)
82. Dong, Fangxiao et al. (2026). *Training-Phase-Aware Optical Circuit Switching Reconfiguration for Large Language Model*. Optical Fiber Communication Conference (OFC) 2026. Abstract only. Evidence that reconfigurable optical (OCS) topologies can accelerate LLM training collectives by matching topology to the communication phase. [doi:10.1364/ofc.2026.m3f.5](https://doi.org/10.1364/ofc.2026.m3f.5)
83. Cho, Sanghun et al. (2023). *Logical/Physical Topology-Aware Collective Communication in Deep Learning Training*. 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA). Abstract only. Demonstrates logical/physical topology-aware AllReduce with computation chaining on an NVLink-based DGX-1, relevant to collective optimization. [doi:10.1109/hpca56546.2023.10071117](https://doi.org/10.1109/hpca56546.2023.10071117)
84. Won, William et al. (2023). *ASTRA-sim2.0: Modeling Hierarchical Networks and Disaggregated Systems for Large-model Training at Scale*. arXiv preprint. Abstract only. Provides an open simulator for interconnect/topology-aware distributed training design-space exploration (NVLink, InfiniBand, collectives, disaggregated memory). [doi:10.1109/ispass57527.2023.00035](https://doi.org/10.1109/ispass57527.2023.00035)
85. Jiang, Ziheng et al. (2024). *MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs*. arXiv (Cornell University). Full text read. Production evidence that communication-computation overlap, network tuning, and observability sustain 55% MFU at 12k+ GPUs - a reference point for interconnect-bound LLM training. [doi:10.48550/arxiv.2402.15627](https://doi.org/10.48550/arxiv.2402.15627)
86. Meng, Qingkai et al. (2025). *Astral: A Datacenter Infrastructure for Large Language Model Training at Scale*. arXiv preprint. Abstract only. Field experience with same-rail network design for LLM training clusters, relevant to datacenter interconnect architecture. [doi:10.1145/3718958.3750521](https://doi.org/10.1145/3718958.3750521)
87. Jouppi, Norman P. et al. (2023). *TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings*. arXiv preprint. Abstract only. Seminal evidence that OCS-based reconfigurable optical interconnects are cheaper and lower power than InfiniBand and sustain ~60% peak FLOPS for LLM training. [doi:10.1145/3579371.3589350](https://doi.org/10.1145/3579371.3589350)
88. Liu, Hong et al. (2023). *Lightwave Fabrics: At-Scale Optical Circuit Switching for Datacenter and Machine Learning Systems*. arXiv preprint. Abstract only. Shows reconfigurable optical lightwave fabrics improve availability and LLM training performance, supporting OCS-based interconnect arguments. [doi:10.1145/3603269.3604836](https://doi.org/10.1145/3603269.3604836)
89. Shou, Chenchen et al. (2025). *InfiniteHBD: Building Datacenter-Scale High-Bandwidth Domain for LLM with Optical Circuit Switching Transceivers*. arXiv preprint. Abstract only. Frames the high-bandwidth-domain (NVLink-domain) scalability/cost/fault tradeoff space for LLM training interconnects. [doi:10.1145/3718958.3750468](https://doi.org/10.1145/3718958.3750468)
90. Feng, Shi et al. (2025). *Mercury: A Reconfigurable Datacenter Network with Collaborative Optical Timeslot Switching and Optical Circuit Switching*. Optical Fiber Communication Conference (OFC) 2025. Abstract only. Evidence for nanosecond-scale reconfigurable optical switching as an interconnect option for ML training. [doi:10.1364/ofc.2025.m4h.3](https://doi.org/10.1364/ofc.2025.m4h.3)
91. Yang, Chongyi et al. (2025). *BEAM: A Multi-Channel Optical Interconnect for Multi-GPU Systems*. 2025 Design, Automation &amp;amp; Test in Europe Conference (DATE). Abstract only. Argues for in-package optical (vs electrical) GPU interconnects for bandwidth and power at scale, relevant to NVLink-alternative roadmaps. [doi:10.23919/date64628.2025.10993197](https://doi.org/10.23919/date64628.2025.10993197)
92. Zhou, Linjie & Luo, Xianshu (2026). *Co-packaged optics: enabling the next generation high-bandwidth and energy-efficient interconnects*. Advanced Photonics Nexus. Abstract only. Points to co-packaged optics as a packaging-level interconnect direction for AI/network systems. [doi:10.1117/1.apn.5.3.030101](https://doi.org/10.1117/1.apn.5.3.030101)
93. Nabavinejad, Seyed Morteza et al. (2020). *An Overview of Efficient Interconnection Networks for Deep Neural Network Accelerators*. IEEE Journal on Emerging and Selected Topics in Circuits and Systems. Abstract only. Background survey on on-chip NoC/interconnect design tradeoffs for DNN accelerators, upstream of scale-out interconnects like NVLink/RoCE. [doi:10.1109/jetcas.2020.3022920](https://doi.org/10.1109/jetcas.2020.3022920)
94. Jung, Myoungsoo (2025). *Compute Can't Handle the Truth: Why Communication Tax Prioritizes Memory and Interconnects in Modern AI Infrastructure*. arXiv preprint. Full text read. Useful synthesis of the communication-tax (35-70% of training time) and the NVLink/CXL/UALink bandwidth hierarchy for LLM infrastructure. [doi:10.48550/arxiv.2507.07223](https://doi.org/10.48550/arxiv.2507.07223)
95. Gouk, Donghyun et al. (2023). *Memory Pooling With CXL*. IEEE Micro. Abstract only. Quantifies CXL.mem vs RDMA for memory disaggregation, relevant to KV-cache/memory pooling for LLM serving. [doi:10.1109/mm.2023.3237491](https://doi.org/10.1109/mm.2023.3237491)
96. Gu, Yufeng et al. (2025). *PIM Is All You Need: A CXL-Enabled GPU-Free System for Large Language Model Inference*. arXiv preprint. Abstract only. Frames memory-bandwidth-bound LLM inference as the driver for high-bandwidth memory and interconnect systems. [doi:10.1145/3676641.3716267](https://doi.org/10.1145/3676641.3716267)
97. Ko, Younghoon et al. (2026). *IBEX: Internal Bandwidth‑Efficient Compression Architecture for Scalable CXL Memory Expansion*. Proceedings of the 40th ACM International Conference on Supercomputing. Abstract only. Addresses bandwidth/capacity limits of CXL memory expanders, an emerging tier for LLM KV-cache offload. [doi:10.1145/3797905.3800521](https://doi.org/10.1145/3797905.3800521)
98. Quinn, Derrick et al. (2025). *Compute-Enabled CXL Memory Expansion for Efficient Retrieval Augmented Generation*. IEEE Micro. Abstract only. Combines CXL memory expansion with near-memory compute for RAG retrieval, relevant to memory-centric LLM inference. [doi:10.1109/mm.2025.3575280](https://doi.org/10.1109/mm.2025.3575280)
99. Fang, Yunhua et al. (2025). *Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System*. arXiv preprint. Full text read. Quantifies headroom in KV-cache placement across HBM/off-package DRAM tiers, informing memory-interconnect design for LLM inference. [doi:10.48550/arxiv.2508.13231](https://doi.org/10.48550/arxiv.2508.13231)
100. Wang, Zixuan et al. (2022). *Enabling Efficient Large-Scale Deep Learning Training with Cache Coherent Disaggregated Memory Systems*. arXiv preprint. Abstract only. Shows cache-coherent interconnects plus disaggregated memory can beat MPI AllReduce for parameter synchronization, relevant to RDMA/CCI tradeoffs. [doi:10.1109/hpca53966.2022.00018](https://doi.org/10.1109/hpca53966.2022.00018)
101. Li, Huaicheng et al. (2023). *Pond: CXL-Based Memory Pooling Systems for Cloud Platforms*. arXiv preprint. Abstract only. Establishes that small CXL pools meet cloud performance goals, informing memory-pooling economics for LLM serving. [doi:10.1145/3575693.3578835](https://doi.org/10.1145/3575693.3578835)
102. Ganjihal, Sanjeev Rao (2026). *Topology-Aware Data Movement for Disaggregated GPU Inference*. arXiv preprint. Full text read. Quantifies the NVLink/InfiniBand/TCP bandwidth hierarchy (72x spread) that disaggregated inference systems must exploit, critiquing uniform-RDMA designs like DistServe/Splitwise/Mooncake. [doi:10.48550/arxiv.2607.28633](https://doi.org/10.48550/arxiv.2607.28633)
103. Zhong, Yinmin et al. (2024). *DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving*. arXiv (Cornell University). Full text read. Canonical prefill/decode disaggregation paper showing placement must account for interconnect bandwidth to minimize disaggregation communication. [doi:10.48550/arxiv.2401.09670](https://doi.org/10.48550/arxiv.2401.09670)
104. Patel, Pratyush et al. (2023). *Splitwise: Efficient generative LLM inference using phase splitting*. arXiv (Cornell University). Full text read. Quantifies throughput/cost gains of phase splitting and the role of fast interconnects (NVLink/InfiniBand) in KV state transfer between phases. [doi:10.48550/arxiv.2311.18677](https://doi.org/10.48550/arxiv.2311.18677)
105. Qin, Ruoyu et al. (2025). *Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving*. ACM Transactions on Storage. Abstract only. Production proof that disaggregated KVCache using spare CPU/DRAM/SSD/NIC resources scales LLM serving, with data-movement implications across interconnects. [doi:10.1145/3773772](https://doi.org/10.1145/3773772)
106. Wu, Bingyang et al. (2024). *LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism*. arXiv (Cornell University). Full text read. Shows elastic sequence parallelism reduces KV-transfer/communication overhead in long-context serving, quantifying the interconnect cost of KV migration across NVLink/InfiniBand. [doi:10.48550/arxiv.2404.09526](https://doi.org/10.48550/arxiv.2404.09526)
107. Liu, Mingxuan et al. (2025). *ServerlessPD: Fast RDMA-Codesigned Disaggregated Prefill-Decoding for Serverless Inference of Large Language Models*. 2025 IEEE International Conference on Web Services (ICWS). Abstract only. Demonstrates RDMA kernel-integrated remote fork as a mechanism for zero-copy KV state migration in disaggregated serverless LLM serving. [doi:10.1109/icws67624.2025.00045](https://doi.org/10.1109/icws67624.2025.00045)
108. Liu, Yuhan et al. (2025). *LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference*. arXiv (Cornell University). Full text read. Provides an open-source KV cache layer spanning GPU/CPU/storage/network tiers, quantifying gains of cross-engine KV movement for PD disaggregation and prefix reuse. [doi:10.48550/arxiv.2510.09665](https://doi.org/10.48550/arxiv.2510.09665)
109. Yang, Jiaqi et al. (2026). *Beyond Prefill-Decode Disaggregation: Dissecting LLM Inference for Heterogeneous Platforms via Dynamic Operator Scheduling*. arXiv (Cornell University). Full text read. Argues PD disaggregation alone is insufficient and models communication/placement costs (all-reduce, gather, scatter) for operator scheduling across heterogeneous interconnects. [doi:10.48550/arxiv.2607.25498](https://doi.org/10.48550/arxiv.2607.25498)
110. Wei, Yuanxin et al. (2024). *Communication-Efficient Model Parallelism for Distributed In-Situ Transformer Inference*. arXiv preprint. Abstract only. Shows block-parallel inference restructuring to cut communication frequency under weak interconnects, relevant to interconnect-constrained distributed inference. [doi:10.23919/date58400.2024.10546617](https://doi.org/10.23919/date58400.2024.10546617)
111. Li, Zhuohan et al. (2021). *TeraPipe: Token-Level Pipeline Parallelism for Training Large-Scale Language Models*. arXiv (Cornell University). Full text read. Exploits token-dimension pipelining to cut communication stalls in model-parallel LLM training; contrasts with allreduce-based Megatron-LM synchronization. [doi:10.48550/arxiv.2102.07988](https://doi.org/10.48550/arxiv.2102.07988)
112. Li, Shenggui et al. (2023). *Sequence Parallelism: Long Sequence Training from System Perspective*. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Abstract only. Positional reference for sequence parallelism (SP) as an interconnect-relevant alternative to tensor parallelism for long-sequence LLM training. [doi:10.18653/v1/2023.acl-long.134](https://doi.org/10.18653/v1/2023.acl-long.134)
113. Brandon, William T. et al. (2023). *Striped Attention: Faster Ring Attention for Causal Transformers*. arXiv (Cornell University). Full text read. Improves load balance of ring-based context-parallel attention (KV communication overlapped with compute), relevant to long-context training communication patterns. [doi:10.48550/arxiv.2311.09431](https://doi.org/10.48550/arxiv.2311.09431)
114. Gu, Diandian et al. (2024). *LoongTrain: Efficient Training of Long-Sequence LLMs with Head-Context Parallelism*. arXiv (Cornell University). Full text read. Combines head-parallel and context-parallel attention to use all inter-node NICs (HDR InfiniBand) efficiently, quantifying SeqAlltoAll vs P2P communication trade-offs. [doi:10.48550/arxiv.2406.18485](https://doi.org/10.48550/arxiv.2406.18485)
115. Ghadia, Ravi et al. (2026). *Untied Ulysses: Memory-Efficient Context Parallelism via Headwise Chunking*. arXiv (Cornell University). Full text read. Shows headwise chunking to shrink all-to-all activation buffers in context-parallel training, addressing the memory side of long-context collective communication. [doi:10.48550/arxiv.2602.21196](https://doi.org/10.48550/arxiv.2602.21196)
116. Jiang, Chenyu et al. (2025). *DCP: Addressing Input Dynamism In Long-Context Training via Dynamic Context Parallelism*. Proceedings of the ACM SIGOPS 31st Symposium on Operating Systems Principles. Abstract only. Motivates dynamic (rather than static) context-parallel partitioning to cut unnecessary communication for variable-length long-context training. [doi:10.1145/3731569.3764849](https://doi.org/10.1145/3731569.3764849)
117. Narayanan, Deepak et al. (2021). *Efficient large-scale language model training on GPU clusters using megatron-LM*. arXiv preprint. Abstract only. Canonical reference on composing parallelisms to reach thousands of GPUs, setting the communication-efficiency baseline that interconnect-aware schedules target. [doi:10.1145/3458817.3476209](https://doi.org/10.1145/3458817.3476209)
118. Wang, Ang et al. (2020). *Whale: A Unified Distributed Training Framework.*. arXiv (Cornell University). Full text read. Demonstrates graph-level placement of collectives (e.g., NVLink-based AllReduce within nodes) as a lever for efficient giant-model training on heterogeneous clusters. [doi:10.48550/arxiv.2011.09208](https://doi.org/10.48550/arxiv.2011.09208)
119. Sensi, Daniele De et al. (2024). *Exploring GPU-to-GPU Communication: Insights into Supercomputer Interconnects*. arXiv preprint. Abstract only. Provides comparative interconnect characterization (multi-Tbps intra-node networks) for GPU supercomputers, informing expectations for large-scale LLM training fabrics. [doi:10.1109/sc41406.2024.00039](https://doi.org/10.1109/sc41406.2024.00039)
120. Li, Conglong et al. (2021). *1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speed*. arXiv preprint. Full text read. Quantifies allreduce-dominated communication on commodity TCP/Ethernet clusters and shows gradient compression as an interconnect-bandwidth mitigation for LLM-scale training. [doi:10.48550/arxiv.2104.06069](https://doi.org/10.48550/arxiv.2104.06069)
121. Li, Xue et al. (2024). *Near-Lossless Gradient Compression for Data-Parallel Distributed DNN Training*. arXiv preprint. Abstract only. Near-lossless gradient compression as a way to relieve interconnect bandwidth pressure in data-parallel LLM-scale training. [doi:10.1145/3698038.3698541](https://doi.org/10.1145/3698038.3698541)
122. Li, Yulong et al. (2024). *Host-driven In-Network Aggregation on RDMA*. IEEE INFOCOM 2024 - IEEE Conference on Computer Communications. Abstract only. Addresses the collision between in-network aggregation (INA) and RDMA reliable connections for DNN training, relevant to RDMA/RoCE-based training clusters. [doi:10.1109/infocom52122.2024.10621230](https://doi.org/10.1109/infocom52122.2024.10621230)
123. Segal, Raz et al. (2022). *Constrained In-network Computing with Low Congestion in Datacenter Networks*. IEEE INFOCOM 2022 - IEEE Conference on Computer Communications. Abstract only. Formal foundation for in-network aggregation (e.g., SHARP-style) deployment, underpinning congestion reduction for distributed ML collectives. [doi:10.1109/infocom48880.2022.9796980](https://doi.org/10.1109/infocom48880.2022.9796980)
124. Zhu, Haowen et al. (2024). *When In-Network Computing Meets Distributed Machine Learning*. IEEE Network. Abstract only. Survey framing INC (programmable-switch aggregation) as an accelerator for distributed ML, contextualizing switch-offload approaches for collectives. [doi:10.1109/mnet.2024.3368138](https://doi.org/10.1109/mnet.2024.3368138)
125. Wang, Guanhua (2023). *Blink: Fast and Generic Collectives for Distributed ML*. SIGCOMM 2023. Full text read. Spanning-tree packing over heterogeneous NVLink/PCIe/NIC topologies as an alternative to NCCL ring collectives, directly relevant to collective design for training. [doi:10.48550/arxiv.1910.04940](https://doi.org/10.48550/arxiv.1910.04940)
126. Urata, Ryohei et al. (2022). *Mission Apollo: Landing Optical Circuit Switching at Datacenter Scale*. arXiv preprint. Abstract only. Production evidence for optical circuit switching in datacenter networks, a candidate technology for future AI-cluster fabric reconfiguration. [doi:10.48550/arxiv.2208.10041](https://doi.org/10.48550/arxiv.2208.10041)
127. Sun, Yan et al. (2023). *Demystifying CXL Memory with Genuine CXL-Ready Systems and Devices*. arXiv preprint. Full text read. Empirical latency/bandwidth characterization of CXL memory expansion (and CXL vs RDMA remote access), informing memory-disaggregation options for LLM inference. [doi:10.48550/arxiv.2303.15375](https://doi.org/10.48550/arxiv.2303.15375)
128. Das Sharma, Debendra et al. (2023). *An Introduction to the Compute Express Link (CXL) Interconnect*. arXiv preprint. Abstract only. Reference survey for CXL as an interconnect (memory expansion, pooling, peer-to-peer), a growing alternative/complement to RDMA for disaggregated memory. [doi:10.48550/arxiv.2306.11227](https://doi.org/10.48550/arxiv.2306.11227)
129. Arelakis, Angelos et al. (2024). *Streamlining CXL Adoption for Hyperscale Efficiency*. arXiv preprint. Full text read. Shows compressed CXL tiers to expand effective memory capacity at hyperscale, addressing memory-capacity constraints in AI inference without extra network traffic. [doi:10.48550/arxiv.2404.03551](https://doi.org/10.48550/arxiv.2404.03551)
130. Agarwal, Saurabh et al. (2024). *SYMPHONY: Improving Memory Management for LLM Inference Workloads*. arXiv preprint. Full text read. KV-cache migration off the critical path as an alternative to stateful offloading, illustrating state-transfer overheads across cluster interconnects (100Gbps Ethernet). [doi:10.48550/arxiv.2412.16434](https://doi.org/10.48550/arxiv.2412.16434)
131. Li, Wanqian et al. (2026). *DWDP: Distributed Weight Data Parallelism for High-Performance LLM Inference on NVL72*. arXiv preprint. Full text read. Replaces layer-wise all-to-all synchronization with async NVLink weight prefetch in MoE inference, quantifying interconnect-centric scheduling on NVL72. [doi:10.48550/arxiv.2604.01621](https://doi.org/10.48550/arxiv.2604.01621)
132. Ardestani, Ehsan K. et al. (2026). *Provisioning to Runtime Optimization of a 100 MW-Scale AI Cluster*. arXiv preprint. Full text read. Production-scale datapoint on AI-cluster networking (2x400Gbps RDMA per GPU) and power/throughput trade-offs for 100MW-class LLM training clusters. [doi:10.48550/arxiv.2605.24461](https://doi.org/10.48550/arxiv.2605.24461)
133. Shah, Aashaka et al. (2023). *TACCL: Guiding Collective Algorithm Synthesis using Communication Sketches*. NSDI 2023. Full text read. Automated synthesis of topology-aware collective algorithms (respecting PCIe/NVLink/IB link constraints), a key technique for optimizing all-to-all/allreduce in LLM training. [doi:10.48550/arxiv.2111.04867](https://doi.org/10.48550/arxiv.2111.04867)
134. Liu, Hao et al. (2023). *Ring Attention with Blockwise Transformers for Near-Infinite Context*. arXiv preprint. Full text read. Foundational ring-attention result showing KV communication fully hidden behind compute; defines the bandwidth-to-FLOPS ratio governing long-context scaling. [doi:10.48550/arxiv.2310.01889](https://doi.org/10.48550/arxiv.2310.01889)
135. Yang, Amy et al. (2024). *Context Parallelism for Scalable Million-Token Inference*. arXiv preprint. Full text read. Shows ring-based context parallelism scales near-linearly to 128 GPUs even on medium/low-bandwidth interconnects, key evidence for long-context inference over commodity fabrics. [doi:10.48550/arxiv.2411.01783](https://doi.org/10.48550/arxiv.2411.01783)
136. Shoeybi, Mohammad et al. (2020). *Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism*. NeurIPS 2020. Full text read. Seminal tensor-parallel design minimizing allreduce count per layer; the communication baseline against which later interconnect-aware LLM training systems are measured. [doi:10.48550/arxiv.1909.08053](https://doi.org/10.48550/arxiv.1909.08053)
137. Ruhela, Amit et al. (2020). *Analyzing and Understanding the Impact of Interconnect Performance on HPC, Big Data, and Deep Learning Applications: A Case Study with InfiniBand EDR and HDR*. 2020 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). Abstract only. Quantifies end-user benefit of moving from InfiniBand EDR to HDR generations, grounding expectations for RDMA fabric upgrades in DL training. [doi:10.1109/ipdpsw50202.2020.00147](https://doi.org/10.1109/ipdpsw50202.2020.00147)