Decoding the technologies of tomorrow, today.
Exploring the breakthrough innovations shaping our world. From AI infrastructure and robotics to biotech, quantum computing, and spatial tech.
Exploring the breakthrough innovations shaping our world. From AI infrastructure and robotics to biotech, quantum computing, and spatial tech.
For years, the basic unit of enterprise computing was easy to understand: a server. A single chassis contained processors, memory, storage, networking components, power supplies, and everything else needed to run an application. That approach worked well because most workloads were relatively self-contained. A database server could have its own storage, an application server could carry its own memory, and adding capacity often meant adding another complete machine.
Large-scale AI has made that model less convenient. Modern training and inference workloads can demand enormous amounts of accelerator compute, memory bandwidth, storage throughput, and network capacity, but those resources do not necessarily grow at the same rate. A cluster may need more GPUs without needing proportionally more local storage, or additional storage bandwidth without requiring another full set of accelerators.
That mismatch is one reason disaggregated infrastructure has become increasingly attractive. Instead of treating each server as a fixed bundle of resources, the architecture separates compute, storage, networking, and, in some designs, memory into resource pools that can be scaled and managed more independently.
A conventional server packages a collection of components into a single system. That makes deployment relatively straightforward, but it also creates a relationship between resources that may not match the workload.
Imagine a cluster built primarily for AI training. The GPUs may be upgraded every few years as new accelerator generations appear, while storage devices, networking equipment, chassis, and power infrastructure can follow different replacement schedules. If all of those components are permanently tied to the same server configuration, changing one resource may create underused capacity elsewhere.
There is also the question of utilization.
One workload may need a large amount of accelerator compute but only moderate storage throughput. Another may be limited by data loading or preprocessing. A fixed server configuration cannot always adapt efficiently to those changing requirements. Some resources can remain underused simply because they are physically attached to the same machine as resources that are already operating near capacity.
At very high accelerator densities, power and cooling become additional concerns. AI accelerators can consume substantial amounts of electrical power, and clusters containing many of them require careful thermal design. Separating components can give engineers more flexibility in arranging power delivery, cooling, networking, and compute hardware rather than forcing every resource into the same chassis.
The underlying issue is not that traditional servers stop working. It is that their fixed resource ratios can become less attractive as AI clusters grow larger and workloads become more specialized.
Disaggregation changes the unit of design.
Instead of asking, “How many identical servers should this cluster contain?” engineers can ask, “How much compute, storage, memory, and networking capacity does the workload actually need?”
Those resources can then be organized into separate pools and connected through high-speed fabrics.
The exact implementation varies. Some systems separate only storage from compute. Others use composable architectures that allow compute, accelerators, memory, and networking resources to be assigned dynamically. The goal is not to separate everything simply for the sake of separation. It is to make each resource easier to scale and replace when its requirements change.
Three components are particularly important: compute, storage, and the network fabric that connects them.
In an AI-oriented disaggregated design, compute infrastructure can be built around dense accelerator systems. These may use GPUs or other specialized AI processors, together with host CPUs, high-speed memory, and the interfaces required to communicate with the rest of the cluster.
The compute layer is optimized primarily for executing workloads. It does not necessarily need to carry large amounts of local storage because training data and model artifacts can be supplied by separate storage systems.
This separation can make hardware refreshes easier. When a new accelerator generation becomes available, operators may be able to replace compute systems without redesigning the entire storage architecture.
There is still a practical limit, of course. The accelerators need sufficiently fast access to memory and data, and some workloads benefit from local storage or caching. Disaggregation does not eliminate those requirements. It simply gives engineers more flexibility in deciding where each resource should live.
AI training can process enormous datasets, so storage performance matters far beyond simple capacity.
A training pipeline may repeatedly read datasets, model checkpoints, metadata, and intermediate files. If the storage layer cannot deliver data quickly enough, expensive accelerators may spend time waiting rather than performing useful computation.
Disaggregated storage addresses this by allowing storage capacity and throughput to be expanded independently from accelerator capacity.
High-performance NVMe drives can be organized into storage systems that are accessed over fast networks. Technologies such as NVMe over Fabrics (NVMe-oF) allow NVMe commands to travel across a network fabric, providing remote access to storage while retaining many of the characteristics that make NVMe attractive.
The advantage is flexibility. A cluster that needs more storage capacity does not necessarily need another complete set of GPU servers. Storage resources can be expanded separately, provided that the network and storage architecture can support the additional traffic.
The trade-off is that remote storage is not identical to locally attached storage. Network latency, congestion, protocol overhead, and the design of the storage system all affect performance. Good disaggregated architectures therefore depend on careful engineering rather than simply moving disks farther away from the compute nodes.
Once compute and storage are separated, the network becomes much more important.
A traditional server can access locally attached resources through internal buses and memory systems. A disaggregated cluster has to move information between physically separate components, which means the network becomes part of the performance equation.
AI clusters commonly use high-performance networking technologies such as InfiniBand or Ethernet-based systems using RDMA, including RoCE. These technologies are designed to provide high bandwidth and low latency for demanding distributed workloads.
The network is not merely a connection between machines. It can determine how efficiently the cluster scales.
This becomes especially important during distributed training, where multiple accelerators may need to exchange model or gradient information repeatedly. Additional latency, congestion, or packet loss can reduce scaling efficiency when communication becomes a significant portion of total execution time.
That is why disaggregated infrastructure depends heavily on the quality of its fabric. Separating resources only makes sense when the connections between them are fast and predictable enough for the intended workloads.
Performance is only part of the argument. Hardware economics matter just as much.
Different resources have different growth patterns. A company may need more storage without needing another large increase in accelerator capacity. Another workload may require additional compute while existing storage is sufficient.
Disaggregation makes it possible to expand the resource that is actually under pressure instead of purchasing a complete server configuration.
That can reduce over-provisioning and make capital spending more closely match workload requirements.
AI accelerators are evolving rapidly, while other infrastructure components may remain useful for much longer.
A compute architecture that allows accelerator systems to be replaced independently can make hardware refreshes less disruptive. Storage shelves, network infrastructure, racks, and power systems may continue operating while the compute layer changes.
This does not guarantee lower costs. Disaggregated systems can require more sophisticated networking and management software, and those components have their own costs. The financial advantage depends on how effectively the infrastructure is utilized.
Disaggregation can also change how failures are handled.
A failure in one component does not necessarily have to make every resource in the surrounding infrastructure unavailable. Redundant storage paths, network links, and compute resources can allow workloads to be redirected when the architecture and software are designed for it.
The exact level of resilience depends on implementation. Disaggregation is not automatically fault tolerant, but separating resources can provide more options for designing redundancy and replacement strategies.
Disaggregation solves one set of problems while introducing another.
The further resources are separated, the more important communication becomes. A workload that performs well with local resources may behave differently when data has to travel across a fabric.
This means architects have to consider topology, bandwidth, latency, congestion, and traffic patterns together. A fast network on paper does not guarantee good application performance.
A traditional server is relatively easy to reason about: the hardware is already assembled, and the operating system sees a fixed collection of resources.
Composable infrastructure is more dynamic. Software has to determine which compute resources belong to a particular workload, where its storage should come from, how those resources are connected, and when they can be released.
That requires orchestration, monitoring, resource scheduling, and automation.
The software layer becomes part of the infrastructure rather than an afterthought.
There is also a risk of overengineering.
A small AI deployment may be perfectly well served by conventional GPU servers with local storage. Introducing a sophisticated resource fabric can add complexity without providing enough benefit to justify it.
Disaggregation becomes more compelling when resource demands are large, uneven, rapidly changing, or expensive to provision independently.
In other words, the architecture should follow the workload rather than the other way around.
The growth of AI is changing how data centers are designed because accelerators have become only one part of a much larger performance equation.
Compute needs to be fed with data. Data needs to move through storage systems. Accelerators need high-bandwidth communication with one another. Power and cooling infrastructure must support dense hardware. Software must coordinate all of it.
That makes the old idea of a server as a self-contained unit less useful for some large-scale deployments.
Disaggregated infrastructure offers another approach: treat compute, storage, memory, and networking as resources that can be combined according to the needs of a workload.
It is not a universal replacement for conventional servers. The additional networking, orchestration, and operational complexity can be substantial. But at large AI scale, the ability to upgrade, expand, and allocate resources independently can be valuable.
The bigger lesson is straightforward. As computing workloads become more specialized, infrastructure does not necessarily need to become more tightly packaged. Sometimes the more effective design is to separate the pieces, connect them with a capable fabric, and let software decide how those pieces work together.
That shift—from the server as the basic unit to the cluster as a pool of composable resources—is one of the more important architectural changes taking place beneath modern AI systems.