rGPU Open-Sourced: Enables Remote NVIDIA GPU Execution for PyTorch Tensors and CUDA Binaries

rGPU

rGPU runs GPU work on a remote NVIDIA machine while the application stays on the client. It currently offers two paths:

The PyTorch device is the simpler integration. The CUDA shim covers existing binaries but has a larger compatibility surface.

Documentation

The Fumadocs site in website/ is the product documentation:

• Quickstart
• Training
• nanoGPT example
• CUDA shim
• Operations
• Configuration reference
• Performance
• Troubleshooting

Engineering records and experiments are indexed in docs/README.md.

Quick start: PyTorch device

Install rGPU with pip install rgpu, or pip install -e ./python from this checkout, then follow the quickstart to deploy the server. Save this as smoke.py in your workload directory:

import torch
import rgpu

x = torch.ones(4, device="rgpu")
print((x * 2).sum().item())  # 8.0

Run it in the environment where rGPU is installed, using your server's SSH destination and options:

rgpu-run --host user@gpu-host --ssh-port 2222 -i ~/.ssh/gpu_key \
  python smoke.py

The program selects the device; rgpu-run opens the tunnel and configures the connection. The expected output is 8.0.

For existing Linux CUDA programs, follow the CUDA shim guide, starting with ./scripts/build_client.sh.

Neither protocol authenticates or encrypts connections. Keep rgpu-opserver on its default localhost bind and use SSH. The CUDA server listens on all IPv4 interfaces: restrict port 9713 with host/cloud firewall rules before starting it, even when using an SSH tunnel. See deployment.

Development

# C++ client and fake-driver tests
./scripts/build_client.sh

# Python tests
python -m pip install -e './python[test]'
python -m pytest python/tests

# Static documentation
npm --prefix website ci
npm --prefix website run build

See scripts/README.md for the remaining build, cloud, and hardware commands. Generated C++ is committed; its policy and regeneration steps are in codegen/README.md.

Repository map

Historical implementation notes and experimental results are indexed in docs/README.md.

作者发帖说明（HN）：

rGPU runs GPU work on a remote NVIDIA machine while the application stays on the client.