Running Qwen3.8-Flash-Next on DGX Spark: 7-Minute Startup and a 128K Context Configuration

Run the startup command, brew a coffee, and return to find the DGX Spark terminal still preparing the model. The first startup takes about seven minutes. But the second startup on the same machine may not be as quick. Reusing the PLE file as-is can stretch startup to 53 minutes.

Qwen3.8-Flash-Next is a 125B-A6B model with an additional 51B PLE. DGX Spark has about 121.63 GiB of usable memory. Even so, the model can run thanks to a design that keeps PLE on SSD. This article covers the configuration we verified in our lab and the issues most likely to cause trouble during startup.

Why It Runs on DGX Spark#

Although Flash-Next carries the Qwen3.8 series name, it is introduced as a preview of the Qwen4 architecture. Its official designation is “125B-A6B”: of its 125B total parameters, roughly 6B are used for each token. It also has a 51B PLE (n-gram embedding tables) and a 4B MTP head for speculative decoding. The configuration described here does not load MTP.

Despite PLE’s size, each token accesses only one row from each of 16 tables, rather than the entire set. With NVFP4, this amounts to about 2.5 KB of reads per token. PLE can therefore reside in a file on SSD, with only the required rows read as needed. About 81 GB of main model weights are loaded into memory, while the PLE backing file occupies about 47.7 GiB.

This differs from moving expert weights to SSD. Experts are used every time a token is generated, so waiting on SSD reads affects speed. PLE is enormous, but the small amount accessed makes this approach viable.

The KV cache also helps. Of the model’s 48 layers, 36 use Gated DeltaNet; only the remaining 12 Qwen Sparse Attention layers have a KV cache. In our measurements, the KV cache occupied 3.04 GB at 128K tokens. This makes it easier to keep cache memory usage manageable when feeding the model long documents.

Model and Environment Used#

The checkpoint we used in our lab was senfu/Qwen3.8-Flash-Next-NVFP4, revision 5d37b3b3711d8406174b96ff950c0aa16324b266. It contained 421 files totaling 128.9 GB. Since the disk must hold both the model itself and the PLE backing file, plan for around 200 GB of free space.

The hardware was DGX Spark (GB10, 128 GB unified memory, 273 GB/s memory bandwidth). Use the open kernel modules from the nvidia-driver-580-open driver family. With the proprietary driver, we encountered illegal memory access during PLE processing. If modinfo nvidia | grep -i license returns Dual MIT/GPL, you have the open driver; if it returns NVIDIA, you have the proprietary driver.

We built SGLang from upstream source at commit 4fb9b5b5bad41ac7de5586ec812c421fe3aa2b8b. The Flash-Next fixes landed upstream on September 5, 2026, while the public Docker image available at the time had been built one day earlier. That is why this configuration uses a source build. The environment used torch==2.13.0 and flashinfer_python==0.6.18. If building the Rust extensions requires cargo, you can skip them with the following setting, since they are unnecessary for this setup.

SGLANG_BUILD_RUST_EXTS=none

Environment Variables to Set Before Startup#

Set these three environment variables before starting the server.

export SGLANG_QWEN4_PLE_FILE_SKIP_DEVICE_CHECK=1
export MAX_JOBS=2
export SGLANG_WARMUP_TIMEOUT=3600

The first variable skips the check SGLang uses to determine whether PLE can reside on SSD. On GB10, the attribute used for this check returns false even with the open driver. However, GB10 can use ATS to provide the required access, and the configuration worked with the check bypassed.

MAX_JOBS=2 limits JIT compilation parallelism. With the default setting, the server sometimes exhausted host memory and exited. SGLANG_WARMUP_TIMEOUT=3600 addresses startup warmup timing out after 600 seconds. Compilation and initialization take time, so we extend the timeout to one hour.

Startup Command for c=1#

The following example limits concurrent requests to one for interactive use. First, adjust the model path to match where you saved it.

python3 -m sglang.launch_server \
  --model-path ~/models/qwen38-flash-next-senfu \
  --host 0.0.0.0 --port 30000 \
  --quantization modelopt_fp4 \
  --ple-offload-embedding --ple-offload-backend file \
  --mem-fraction-static 0.85 \
  --context-length 32768 --max-total-tokens 32768 \
  --chunked-prefill-size 1024 \
  --max-running-requests 1 --max-mamba-cache-size 1 \
  --disable-radix-cache \
  --watchdog-timeout 3600 \
  --prefill-attention-backend triton --decode-attention-backend trtllm_mha \
  --reasoning-parser qwen3 --tool-call-parser qwen3_coder

We measured 27 tok/s at c=1 in our lab. In the 128K context measurement, the KV cache occupied 3.04 GB. The startup example above uses a 32K context to make it easier to adjust settings while monitoring memory. To use 128K, change --context-length and --max-total-tokens, and check memory usage under your actual workload.

The quantization option is --quantization modelopt_fp4. Although the model card recommends modelopt_mixed, this article uses the configuration we successfully launched with the option shown here.

The Pitfall That Can Make the Second Startup Take 53 Minutes#

The PLE backing file is created on the first startup and remains after a restart. Writing to an existing file again sometimes caused startup to take about 53 minutes. In our lab, we applied a patch that samples the file contents at 32 locations using 4 KB windows and skips writing if the contents are correct. This brought subsequent startups back down to about seven minutes.

Without the patch, deleting the old backing file before startup also avoids this issue. Check your SGLang configuration to confirm the file’s location before deleting it. Take care not to confuse it with the model weights.

We initially also tried a patch that moved PLE row retrieval to the CPU, but its purpose was to work around the proprietary driver issue. It was unnecessary after switching to the open driver.

What to Check First#

If startup stalls, first check which driver you are using. Next, confirm that the three environment variables are set and that the checkpoint revision and SGLang commit match. Finally, check whether reusing the PLE backing file is causing the slowdown.

With this configuration, we ran Flash-Next on DGX Spark and measured 27 tok/s at c=1. Start with the 32K settings, then expand the context to suit your prompt lengths and available memory.