Glass-panel workstation with a graphics card and cooler, monitor with training charts beside it

An AI workstation has two memory pools. The GPU carries its own memory, and the workstation also carries system RAM on the motherboard. Most sizing advice talks about GPU memory, but system RAM is the pool people under buy, and it is the one that is hardest to fix after the fact. This guide gives you a way to size it from your own workload, using published rules and simple arithmetic rather than a guess.

The short answer

  • Start with a floor of twice your total GPU memory. Puget Systems recommends system RAM of at least double the total GPU memory, so two GPUs with 32GB between them call for at least 64GB.
  • Then add what your work puts in RAM: a dataset held in memory, offloaded model data, and the model itself if it runs on the CPU.
  • Model size is parameters times bytes per parameter. That one formula tells you how much memory a model needs at each precision.
  • Buy to fill every memory channel evenly, on a platform that has the capacity you need.

What system RAM does in an AI workstation

The GPU does the math in its own memory, but it depends on system RAM for several jobs. Data is read, decoded and prepared in system memory before it reaches the GPU. Frameworks can pin memory, which the operating system cannot swap out, to speed up copies to the GPU. When a model does not fit in the GPU, libraries can place part of it in system RAM. The AI server RAM guide covers the same ideas for rack systems.

Size the model first

A model's weights take its parameter count multiplied by the bytes used for each parameter. Lower precision uses fewer bytes, so a quantized model is smaller. The numbers below are for the weights alone, using a hypothetical 7 billion parameter model.

Precision Bytes per parameter Weights for a 7 billion parameter model
float32 4 28 GB
float16 2 14 GB
int8 1 7 GB
int4 0.5 3.5 GB

Hugging Face's Accelerate library includes an estimate-memory command that reports the loading size for a real model, and its documentation says to expect up to an additional 20% for inference. For the 7 billion parameter example in float16, that is 14GB of weights and up to about 17GB in use.

Loading can need more than the model size

Hugging Face's documentation says that loading a large model normally needs about twice its size in memory, because the library builds an empty model and then fills it with the pretrained weights. Its low_cpu_mem_usage setting and its device_map setting bring the peak close to one copy of the model. If you cannot use them, plan for two copies in system RAM while the model loads.

Training needs several times the weights

Training holds more than the weights. Hugging Face's model memory guide lists, for mixed precision training with Adam, 6 bytes per parameter for the weights, 4 for the gradients and 8 for the optimizer states, which adds up to 18 bytes per parameter on the GPU. Offloading libraries can move the optimizer states to system RAM, which shifts that part of the load from GPU memory to the system RAM you installed.

Match the workload to the memory

Workload What fills system RAM How to size it
Run a quantized model locally The model weights, plus working space Model size in memory, plus up to 20% for inference, and the 2x GPU memory floor
Run a model on the CPU only All of the weights live in system RAM Parameters x bytes per parameter, plus up to 20%
Fine tune or train with GPUs Data loading, pinned buffers, any offload The 2x GPU memory floor, plus offloaded states and buffers
Data science and analytics The dataset held in memory The size of the dataset, plus working copies
Multi GPU workstation The total across every GPU Add up all GPU memory first, then apply the floor

How to size system RAM

  1. Add up your GPU memory. Multiply memory per GPU by the number of GPUs.
  2. Apply the floor. Take twice that number as the minimum system RAM.
  3. Add your dataset if you keep it in memory. Puget Systems notes this can reach about 1TB, and occasionally more, for data analysis.
  4. Add model and offload needs. For a CPU run, use parameters times bytes per parameter plus up to 20%. For offload, add the offloaded states.
  5. Leave room for the system. The operating system, your tools and pinned buffers all share the same pool.
  6. Divide by the channels. Split the total across all memory channels with identical modules, then round up to the next size the board supports.

This produces a range with a clear reason behind each part of it. If you can run a test job, watch real memory use and size above the peak.

Choose a platform with the capacity

Mainstream desktop boards take unbuffered DIMMs and stop at a lower capacity than workstation platforms. Puget Systems notes that workstation and server processors support far more system memory than consumer chips, and that the Xeon W, Xeon 600 and Threadripper PRO platforms have 8 memory channels. AMD lists the WRX90 platform for Threadripper PRO 9000 WX-Series at 8 channels, RDIMM, up to 2TB. Intel lists the Xeon w9-3595X at 8 channels, DDR5-4800, ECC supported and up to 4TB of memory. Always confirm the limit in your own board's manual, since a board can stop short of the processor's maximum.

With 8 channels, install 8 identical modules to use every channel. Your total capacity divided by 8 is the module size you need. The compatible RAM guide shows how to read a board's memory specification.

ECC and module type

  • ECC. Long training and analysis jobs can run for days, and ECC memory corrects single bit errors along the way. The ECC vs non-ECC guide explains the difference.
  • RDIMM or UDIMM. Workstation platforms with 8 channels use registered DIMMs (RDIMM), while smaller workstations use ECC or non-ECC unbuffered DIMMs (UDIMM). The memory types explained guide covers each type.
  • Speed. Speed matters most for work that runs on the CPU. For GPU bound work, capacity and channel balance matter more. Puget Systems notes that 8 memory channels can significantly affect CPU bound workloads.

Common mistakes

  • Matching system RAM to GPU memory one for one, which falls below the 2x floor.
  • Sizing for the model and forgetting the load step, which can briefly need two copies.
  • Filling only some channels, which leaves bandwidth unused.
  • Buying memory before checking the board's capacity limit.
  • Mixing module types or sizes in the same channel set.

Find memory for your workstation

Match the module to your platform: DDR5 RDIMM, DDR4 RDIMM, DDR5 ECC UDIMM and DDR4 ECC UDIMM. If you are building on a branded workstation, see the workstation memory guide, the Dell Precision memory guide and the Lenovo workstation memory guide. For a two processor board, read single vs dual socket memory. If your system is not listed, send us the model name and we will check it.

Frequently asked questions

How much RAM does an AI workstation need?

At least twice the total GPU memory, as a floor, plus whatever your workload keeps in system RAM, such as a dataset or offloaded model data. Add those up for your own job, because the answer changes with the workload.

Can system RAM make up for a small GPU?

Partly. Hugging Face's device_map setting can spread a model across GPUs, the CPU and disk, starting with the fastest device. A model that spills into system RAM runs, but the GPU no longer holds all of it, so plan GPU memory for the model first.

Does RAM speed matter for AI work?

It matters most for CPU bound work. For GPU bound work, capacity and a balanced channel layout matter more than the last step of speed.

Do I need ECC memory for AI?

It is not required for the software to run, but long jobs benefit from error correction, and workstation platforms such as the Xeon w9-3595X support ECC memory. Check that your processor and board both support it.

Does NEMIX RAM supply memory for AI workstations?

Yes. We list DDR5 and DDR4 RDIMM, ECC UDIMM and other module types matched to the workstation and server systems that list them. Send us your board or system model and we will confirm the modules.

Sources

Puget Systems AI development hardware recommendations, Hugging Face model memory anatomy, Hugging Face big models guide, Hugging Face Accelerate model size estimator, AMD Ryzen Threadripper workstation processors and Intel Xeon w9-3595X specifications, read on October 29, 2026. Specifications change, so confirm against your board maker's documentation.

Buying memory for workstation fleets

Purchase orders, net terms and volume pricing are available for engineering, reseller and public-sector orders. Request a quote with your models and quantities, or contact our sales team.

NEMIX RAM is an independent memory supplier and is not affiliated with the processor, board or system makers named here. Product and brand names are trademarks of their respective owners, used only to identify compatible systems.

Ai workstation, Ddr5 rdimm, Ecc memory, Guide, Machine learning, Threadripper pro, Workstation memory, Xeon w

Leave a comment

All comments are moderated before being published