Skip to main content


KV cache offload normally routes every retrieved byte through host RAM on its way to the GPU. GPUDirect Storage moves it from NVMe straight into GPU memory instead. We benchmarked both paths on a 100K-token serving workload on the Solidigm D7-PS1010.

The 2.9x speed-up is what you notice first; the host bandwidth it frees is what scales as the box gets busier.

GDS restored a 100K-token cache 2.9x faster than the conventional POSIX path while using 5x less host memory bandwidth, returning roughly 7.5 GB/s to the rest of the system.

What we found

Retrieval latency
GDS restores a 100K-token cache in 1,725 ms against 4,952 ms on a POSIX O_DIRECT path, a 2.9x reduction, median of 20 repeat requests.
Where the saving lands
Host DRAM bandwidth during retrieval falls from 9.41 GB/s to 1.85 GB/s, a 5.1x reduction, because the bytes stop transiting host memory.
Delivery rate
GDS lands 4.82 GB/s into GPU memory against 2.04 GB/s on the POSIX path. The host-side copy is the bottleneck, not the Solidigm D7-PS1010 underneath it.

Offloading KV cache to SSD is now a standard way to avoid re-prefilling long contexts, and the case for it is well established: a 100K-token session costs seconds of GPU time to rebuild and a few gigabytes to store. What is less settled is which path the bytes take on the way back, and whether that choice is worth configuring deliberately.

There are two. The conventional path reads from the filesystem into a host buffer, then copies that buffer into GPU memory: two hops, both through DRAM. GPUDirect Storage1 replaces them with a single peer-to-peer DMA from the NVMe device into GPU memory across PCIe, with no host bounce. Its value is not primarily latency but resource, since skipping the bounce frees host memory bandwidth and CPU cycles for everything else on the box. LMCache exposes both paths2, which makes them directly comparable. This evaluation measures both at 100K-token scale.

What we ran

Every configuration serves the same 40-request sequence: 20 unique long prompts, then 20 repeats. Each repeat is a genuine restore, since the session has been evicted from GPU memory before it is re-requested. The model is served under vLLM3 at tensor-parallel size 2, with LMCache handling KV offload to its local-disk L2 backend and cuFile driving the GDS path. Reported metrics are medians across the 20 repeat windows, with disk, DRAM, PCIe and CPU samplers clipped to each request's start and end timestamps.

ModelQwen3-Coder-30B-A3B-Instruct-FP8 (30B MoE / 3B active), tensor-parallel size 2
Hardware2x RTX PRO 6000 Blackwell Server Edition
StorageSolidigm D7-PS1010 (TLC, 15.36 TB), XFS, shared by both tensor-parallel ranks
StackvLLM 0.26.0, LMCache 0.5.3rc2, cuFile / GDS
KV backendLMCache local-disk L2; offload chunk size 1024 (GDS) and 256 (O_DIRECT)
Workload40 requests: 20 unique prompts of 98,058-100,022 tokens, plus 20 repeats
MethodMedians across 20 repeat windows

At roughly 9 GB of KV state per session, restore time is dominated by moving bytes rather than by per-request overhead. That is the regime where the data path is worth arguing about; at a few thousand tokens it is not.


Insight 01

Both paths read directly from the Solidigm D7-PS1010.

Both paths read directly from the Solidigm D7-PS1010 during the restore. The POSIX O_DIRECT path sustains 1.97 GB/s and GDS 4.74 GB/s, both comfortably inside the drive's rated throughput4, so the SSD is doing genuine work on every repeat rather than being served from memory.

Figure 1. Physical disk read during retrieval

Bytes read from the device inside each repeat window.

O_DIRECT
1.97 GB/s
GDS
4.74 GB/s

Both paths read from the same Solidigm D7-PS1010. Preliminary results.


Insight 02

GDS restores a 100K-token session 2.9x faster.

GDS restores the evicted 100K-token session in 1,725 ms against 4,952 ms on the O_DIRECT path, both reading the same payload from the same drive. The gap is a property of the data path itself, not of tuning around it.

Figure 2. Time to restore a 100K-token session

End-to-end, median of 20 repeat requests. Shorter is better.

O_DIRECT
4,952 ms
GDS
1,725 ms2.9x faster

Both paths read the same payload from the same drive. Preliminary results.


Insight 03

The saving lands in host memory bandwidth.

Latency is the visible number, but the resource result matters more. During retrieval the O_DIRECT path drives 9.41 GB/s of host DRAM traffic, while GDS drives 1.85 GB/s for the same restore. The O_DIRECT path still stages every byte through a host buffer before the copy to GPU memory; only the peer-to-peer path removes that transit.

On an idle box that bandwidth is effectively free. On a box running other work it is contended, and the 7.5 GB/s returned to the rest of the system is the substantive result rather than the 3.2 seconds of latency.

Figure 3. Host DRAM bandwidth consumed during retrieval

Total memory bandwidth inside each repeat window. Shorter is better.

O_DIRECT
9.41 GB/s
GDS
1.85 GB/s5.1x less

Sampling is coarse relative to the window; see caveats. Preliminary results.


Insight 04

The host copy is the bottleneck, not the drive.

If the Solidigm D7-PS1010 were the limit, both paths would deliver bytes to the GPU at similar rates and differ only in how long they took to start. They do not. GDS lands 4.82 GB/s into GPU memory against 2.04 GB/s on the O_DIRECT path, reading the same data from the same drive. The staging copy through host memory is what caps the conventional path, not the SSD.

Figure 4. Delivery rate into GPU memory

PCIe receive, both tensor-parallel ranks summed. Longer is better.

O_DIRECT
2.04 GB/s
GDS
4.82 GB/s2.4x more

Higher delivery rate means the link is occupied for less wall-clock time. Preliminary results.


Insight 05

Higher CPU rate, but 41% less CPU consumed.

While the restore is running, GDS uses more CPU per second: 3.20 cores against 1.91 on the O_DIRECT path. But it runs for far less time, so the CPU actually consumed per request is lower: 5.53 core-seconds against 9.37, a 41% reduction. GDS runs hotter for 1.73 s instead of cooler for 4.91 s, and the product is what the workload pays for.

The distinction changes how the figure is read. The per-second rate is what would govern how many concurrent streams a single box can hold; the per-request total is what shows up on the bill for a given amount of work.

Figure 5. Host CPU consumed per request

Core-seconds of host CPU per request, over the full repeat phase (87 and 33 samples at 99.9% busy). Shorter is better.

O_DIRECT
9.37 core-s
GDS
5.53 core-s41% less

GDS draws more CPU per second (3.20 cores against 1.91) but for far less time, so it consumes 41% less host CPU per request. Preliminary results.


The numbers

Table 1. Measured medians, 20 repeat requests per configuration

Same drive, same workload, same model.

CONFIGURATIONRESTOREDISK READDRAM BWPCIE RXCPU / REQ
O_DIRECT4,952 ms1.97 GB/s9.41 GB/s2.04 GB/s9.37 core-s
GDS1,725 ms4.74 GB/s1.85 GB/s4.82 GB/s5.53 core-s

CPU / REQ is host CPU consumed per request, in core-seconds. Cold prefill TTFT was 11,201 ms and 9,992 ms respectively; see caveats. Preliminary results.


So, should you turn it on?

Yes, for long-context workloads with real reuse where the host is doing other work. The latency gap is the easy sell, but the durable argument is the 7.5 GB/s of memory bandwidth returned to the rest of the machine.

No, if your working set fits comfortably in RAM. In that case the page cache will outrun both paths, and you are not storage-bound at all.


The takeaway

For long-context serving with genuine reuse, the path between the drive and the GPU is worth configuring deliberately. Both paths read from the same Solidigm D7-PS1010 at multiple gigabytes per second, so the drive is not the limiting factor; the cost of the conventional path is the copy that stages every byte through host memory on the way to the GPU. GPUDirect Storage removes that hop, restoring an evicted 100K-token cache 2.9x faster and returning roughly 7.5 GB/s of host memory bandwidth to the rest of the system. The latency improvement is the visible result; the reclaimed host bandwidth is the one that compounds as more work runs on the same machine.


Caveats and dependencies

ScopeOne model, one context length, one drive, one otherwise-idle machine. No multi-tenant contention was tested in this round, which is precisely where the DRAM saving would matter most.
Parameter asymmetryThe GDS configuration used a chunk size of 1024 against 256 on the O_DIRECT path, chosen from earlier measurement of per-file registration overhead. This is not a matched-parameter comparison and some of the gap may be attributable to it.
DRAM samplingThe memory-bandwidth counter samples at roughly 1.2 s against retrieval windows of 1.7-5.0 s, so those figures rest on one to three samples per window. Treat them as directional.
Upstream cache3 of 20 repeats in every configuration were served by the GPU prefix cache with no storage access. They are included in the medians and affect both configurations equally.
Latency captureOn the fastest GDS responses the client did not record a first-token timestamp. End-to-end latency is used throughout and is unaffected.
Cold prefillGDS showed roughly 11% lower cold-prefill TTFT. Cold prefill is compute-bound and should not depend on the storage path; this is unexplained and is not claimed as a GDS benefit.
CPU rateThe per-second CPU figure, higher for GDS, would govern how many concurrent streams a single box can hold. This sequential test cannot measure that; a concurrency test is the right follow-up.

References

1.NVIDIA. "Magnum IO GPUDirect Storage Overview Guide," r1.18. docs.nvidia.com/gpudirect-storage/overview-guide
2.LMCache. "GDS Backend," LMCache documentation. docs.lmcache.ai/kv_cache/gds.html
3.vLLM. "Automatic Prefix Caching," vLLM documentation. docs.vllm.ai/en/latest/features/automatic_prefix_caching.html
4.Solidigm. "D7-PS1010 and D7-PS1030 Product Brief." Rated up to 14.5 GB/s sequential read, 9.3 GB/s sequential write, TLC, capacities to 15.36 TB.

Disclaimer

A preliminary technical evaluation, for informational purposes only. Not a benchmark, product announcement, or performance guarantee. Measurements were made by Metrum AI in a single test configuration and are reported as observed behaviour, not as claims of throughput, latency or availability in other environments; results will vary with model, workload and deployment. The caveats above bound the conclusions. Solidigm is a trademark of Solidigm. RTX and GPUDirect are trademarks of their respective owners. LMCache and vLLM are open-source projects. Qwen is a trademark of Alibaba Group. All other trademarks belong to their respective owners, and their use does not imply endorsement.