General Tech
Serving GLM-5.3 at 607.8 tok/s across two Nvidia DGX stations with vLLM
This tutorial provides a guide on how to serve the GLM-5.3 model on two NVIDIA DGX Stations with vLLM, using tensor parallelism to split the model's weights and computation across both GPUs. The model has 753 billion parameters and 40 billion activated per token, requiring 464 GB of memory, exceeding the capacity of a single GB300 GPU's 252 GB of HBM3e. The tutorial uses two DGX Stations connected over ConnectX-8 RDMA to serve the model, with Station A hosting the API and Station B running a worker with no API server.
Read the full article at developer.nvidia.com →