AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: What Connects Dies And HBM Stacks Inside An AI Chip? on ThorstenMeyerAI.com

Buying for a business?Offer from Amazon

Get business pricing on monitors, keyboards and dev gear

  • Business-only prices and quantity discounts
  • Tax-exempt purchasing
  • Multiple users, one account, clear invoices
As an affiliate, we earn on qualifying purchases.

TL;DR

A current AI accelerator is a package of compute dies, high-bandwidth memory stacks and an interposer that links them, rather than a single piece of silicon. The source explainer says memory bandwidth and advanced packaging are increasingly important constraints on accelerator performance and supply, but some product specifications and supply forecasts are manufacturer claims.

A November 22 explainer from ThorstenMeyerAI.com describes how silicon interposers connect AI accelerator compute dies with high-bandwidth memory (HBM), showing why packaging is part of the performance and production equation. The account draws on company specifications and a schematic 3D model; it says the precise dimensions of the modeled packages and dies are not publicly specified by Nvidia or AMD.

The package is assembled from several layers. A package substrate carries power and signals to the module, while a silicon interposer above it provides dense wiring between the compute dies and HBM stacks. HBM consists of DRAM chips stacked vertically and connected using through-silicon vias, or TSVs. The close connections allow many signals to run between memory and processors in a compact area.

The explainer cites Nvidia’s product information for a 19.2 terabytes-per-second memory bandwidth figure for Rubin, compared with 900 gigabytes per second for the V100. These are figures for different Nvidia GPU generations, not a controlled performance comparison. Nvidia says its Rubin GPU has two reticle-limited compute dies and 336 billion transistors. AMD describes its MI350-series design as eight vertically stacked accelerator complex dies and two I/O dies; AMD gives the MI355X a total of 185 billion transistors.

Packaging technology determines how large and complex a connected assembly can be. TSMC says its CoWoS-S packaging can support an interposer up to about 2,700 square millimeters, or 3.3 times the maximum reticle area. The source says TSMC has put a five-reticle-size CoWoS-L design into real-world production, citing a company executive on October 15, 2026. CoWoS-L connects smaller silicon bridges in a carrier layer rather than relying on one large silicon interposer.

At a glance
reportWhen: Published November 22, 2026
The developmentA November 22 explainer details how silicon interposers connect AI accelerator compute dies to HBM stacks, and why packaging capacity can constrain production.

Why Memory Packaging Limits AI Capacity

Compute dies can perform calculations only as quickly as data reaches them. HBM bandwidth helps keep those processors supplied with data, while the interposer provides the short, dense connections needed between memory and compute. This makes memory placement and package design consequential to accelerator throughput, not merely details of how a chip is assembled.

Packaging also affects how many accelerators manufacturers can deliver. The source identifies advanced packaging as a production bottleneck, alongside memory supply. SK hynix has said demand for memory will exceed supply through 2030; that is a company forecast, not an independently established outcome. If supply remains tight, it could limit the number of systems available to data centers and businesses, even when demand for AI computing is strong.

The system-level scale underscores the stakes: Nvidia’s GB200 and GB300 NVL72 rack designs contain 72 GPUs and 36 CPUs, according to the cited documentation. A rack depends on far more than accelerator silicon, including HBM, packaging, networking, power delivery and liquid cooling. The explainer’s central point is that constraints at any of these levels can affect how quickly AI compute capacity expands.

De'Longhi Magnifica Evo Automatic Espresso Machine with Milk Frother, Silver

De'Longhi Magnifica Evo Automatic Espresso Machine with Milk Frother, Silver

Bose QuietComfort Wireless Bluetooth Noise-Cancelling Headphones, Ice Blue

Bose QuietComfort Wireless Bluetooth Noise-Cancelling Headphones, Ice Blue

As an affiliate, we earn on qualifying purchases.

From HBM Towers to Rack Systems

The source’s 3D model presents the package from the bottom up, but its dimensions are schematic. It says neither Nvidia nor AMD publishes the sizes in millimeters of the relevant dies, full packages or HBM stacks, so the model does not estimate them. The layer counts and ordering are intended to illustrate construction, not provide a scale drawing.

TSVs are vertical electrical connections through stacked DRAM. HBM stacks can combine multiple memory layers above a base die; JEDEC’s HBM4 standard, published in April 2025, permits stacks of four, eight, 12 or 16 layers and up to 64 gigabytes per stack. SK hynix and Samsung have described thinner layers and tighter spacing in their products, but those statements are manufacturer specifications and announcements.

The assembly is cooled from above with a liquid-cooled plate. Nvidia’s documentation says coolant runs through cold plates attached to CPUs and GPUs in the tray. For a GB200 NVL72 rack, Nvidia lists 120 kilowatts of cooling capacity. The rack example puts the package in a larger setting: compute trays, NVLink switch trays, management switches and power shelves all work together to form the system.

Package Dimensions and Supply Outlook

The explainer does not provide measured dimensions for the dies, packages or memory stacks. Its model uses schematic sizes, so readers should not treat its visual proportions as product measurements. Some technical details are based on vendor product pages and statements, rather than independent testing.

It is also unclear from the source how much packaging capacity is available across the industry, how production is divided among accelerator designs, or whether supply forecasts will hold. The cited bandwidth figures describe product specifications; they do not establish the performance users will see in every workload. The source does not quantify how much of any current AI-compute shortage is attributable specifically to HBM, packaging, or other constraints.

Capacity Depends on the Next Packages

The next developments to watch are manufacturers’ progress in producing larger advanced packages, the move to taller HBM stacks and the availability of memory for accelerator products. TSMC’s reported work on CoWoS-L packages larger than five reticle areas could expand package design options, but the source gives no production volume or delivery schedule for its stated work on nine- and 14-reticle sizes.

As Nvidia and AMD bring new accelerator generations to market, their specifications and system documentation should clarify how compute dies, HBM capacity, bandwidth, cooling and rack configurations change. For now, the source’s package diagrams explain the connection between the components; they do not establish when supply constraints will ease or what effect they will have on prices and deployment schedules.

Key Questions

What connects compute dies to HBM stacks?

A silicon interposer carries dense wiring between the compute dies and HBM stacks inside the package. The package substrate below it connects the assembly to the larger module.

What does HBM mean?

HBM means high-bandwidth memory. It is DRAM arranged in vertical stacks, with layers linked by through-silicon vias.

Why do AI accelerators use stacked memory?

Stacking memory close to compute provides many short connections in a compact area. That can support high memory bandwidth, helping supply data to the accelerator’s compute dies.

Are the dimensions in the explainer’s model exact?

No. ThorstenMeyerAI.com says the model shows layer order and counts with schematic sizes. It says Nvidia and AMD do not publish the relevant dimensions in millimeters.

Does the source establish when memory shortages will end?

No. It cites SK hynix’s forecast that demand will exceed supply through 2030, but that is a company outlook. The source does not provide an independently confirmed date for when supply will meet demand.

Source: ThorstenMeyerAI.com

You May Also Like

One-idea-per-email drip platform for developer onboarding

A developer-relations lead plans to test a new email onboarding tool that delivers one technical idea per message, aiming to improve activation rates.

Storage Systems: The Weight‑Distribution Mistake That Wrecks Handling

Correct weight distribution is crucial for safe storage systems, but neglecting it can lead to catastrophic failures and costly repairs if you don’t read more.

Developing Level‑4 Autonomous Bus Sensors: LiDAR, Radar, and Cameras

Creating reliable Level-4 autonomous bus sensors involves integrating LiDAR, radar, and cameras—discover how calibration and data fusion unlock safer autonomous transportation.