Termsslug: vllm

vLLM

A high-throughput and memory-efficient serving engine for large language models.

Curated Intelligence

4 items · Page 1/1
IT之家 · 智能时代T2·Open Source
78 pts
StepFun Open-Sources Step 5 Preview Weights with Day-0 Support from Huawei Ascend and Other Domestic Chips

StepFun has open-sourced the Step 5 Preview model weights, enabling deployment via vLLM, SGLang, and Transformers. Major domestic chipmakers including Huawei Ascend, MetaX, Biren, and Moore Threads announced Day-0 support, with Huawei leveraging its Atlas 950 SuperPoD to achieve high-efficiency inference optimization for trillion-parameter models.

Hugging Face BlogT1·Tools
48 pts
Run vLLM Server on HF Jobs in One Command

Hugging Face enables one-command deployment of private, OpenAI-compatible vLLM endpoints on HF Jobs via the `huggingface_hub` CLI. This serverless, pay-per-second solution eliminates infrastructure provisioning for quick testing and batch generation, complementing the production-oriented Inference Endpoints.

智谱 AI 研究T1·Models
78 pts
Zhipu Open-Sources GLM-OCR: 0.9B Model Tops Doc Parsing SOTA

Zhipu AI has released and open-sourced GLM-OCR, a lightweight model with only 0.9B parameters that achieves state-of-the-art performance (94.6 score) on OmniDocBench V1.5. Optimized for complex scenarios like handwriting, tables, and code, it supports efficient deployment via vLLM and SGLang, offering a high-performance solution for edge computing and high-concurrency applications.

Hugging Face BlogT1·Tools
35 pts
TRL Integrates Co-located vLLM for Efficiency

The Hugging Face TRL library introduces a co-located vLLM architecture to optimize LLM training and inference workflows. This update aims to improve overall compute utilization by sharing GPU resources to reduce memory fragmentation. Specific implementation details, performance benchmarks, and applicable scenarios remain unclear based solely on the headline.