StepFun has open-sourced the Step 5 Preview model weights, enabling deployment via vLLM, SGLang, and Transformers. Major domestic chipmakers including Huawei Ascend, MetaX, Biren, and Moore Threads announced Day-0 support, with Huawei leveraging its Atlas 950 SuperPoD to achieve high-efficiency inference optimization for trillion-parameter models.
vLLM
A high-throughput and memory-efficient serving engine for large language models.
Curated Intelligence
Hugging Face enables one-command deployment of private, OpenAI-compatible vLLM endpoints on HF Jobs via the `huggingface_hub` CLI. This serverless, pay-per-second solution eliminates infrastructure provisioning for quick testing and batch generation, complementing the production-oriented Inference Endpoints.
Zhipu AI has released and open-sourced GLM-OCR, a lightweight model with only 0.9B parameters that achieves state-of-the-art performance (94.6 score) on OmniDocBench V1.5. Optimized for complex scenarios like handwriting, tables, and code, it supports efficient deployment via vLLM and SGLang, offering a high-performance solution for edge computing and high-concurrency applications.
The Hugging Face TRL library introduces a co-located vLLM architecture to optimize LLM training and inference workflows. This update aims to improve overall compute utilization by sharing GPU resources to reduce memory fragmentation. Specific implementation details, performance benchmarks, and applicable scenarios remain unclear based solely on the headline.