Tencent Open-Sources Hy-SimulMT: 3s Latency Beats Human Interpreters
Tencent has open-sourced Hy-SimulMT, a simultaneous interpretation LLM supporting real-time Chinese-English speech-to-speech translation with voice cloning. With a latency of ~3 seconds, it outperforms the average human interpreter and matches or exceeds expert-level translation quality. The model is now available for free via the Tencent Hy Translation app.
Full Report
Tencent Hunyuan Simultaneous Interpretation LLM Hy-SimulMT Open-Sourced, Achieves 3-Second Latency Surpassing Human Interpreters
On October 26, it was announced that Tencent's Hunyuan simultaneous interpretation large model, Hy-SimulMT, has been released and open-sourced. The model features low latency and high accuracy, supporting real-time speech-to-speech translation between Chinese and English. Speaker audio can be input into the Hy-SimulMT model in real time; the model translates while listening, providing immediate text and voice output. It also supports real-time voice cloning, highly replicating the speaker's voice and tone. The model debuted on the Tencent Hy Translation App, offering free service experiences.
According to official information obtained by IT Home, low latency is a key highlight of the model. The response time of the Hunyuan simultaneous interpretation large model Hy-SimulMT is measured at approximately 3 seconds, compared to an average of 4–5 seconds for professional human interpreters. Evaluation results show that this model's translation performance surpasses mainstream open-source simultaneous interpretation models available on the market, and even exceeds most commercial simultaneous interpretation APIs.
The officials stated that on RealSI, a commonly used long-audio open-source evaluation set in the industry, Hy-SimulMT demonstrated strong simultaneous interpretation performance, as shown in the figure below:
The horizontal axis represents StreamLAAL, a latency metric for simultaneous interpretation, indicating the delay in seconds between the model's translation output and the input audio (lower is better). The vertical axis represents translation quality, scored using GPT6.1-Sol (higher is better). Hy-SimulMT achieves the best performance among open-source models—and even surpasses most commercial APIs—while maintaining the lowest latency.
Hy-SimulMT also performs well on short-audio test sets, as shown in the figure below:
To more accurately assess the true capabilities of Hy-SimulMT, the team widely collected newly emerged Chinese and English audio from the internet within compliance limits. This data covers ten domains: software AI, medicine, manufacturing & energy, natural sciences, agriculture & food, business management, law, education, culture & tourism, and sports & lifestyle, involving multiple scenarios such as speeches, interviews, and meetings.
Using this dataset, the team not only tested the effectiveness of Hy-SimulMT but also invited multiple human simultaneous interpretation experts to interpret these audio samples.
There were six human simultaneous interpretation experts in total: three hold Level 1 CATTI Interpreter Certificates, and three hold Level 2 CATTI Interpreter Certificates. All experts have years of practical experience in simultaneous interpretation. The evaluation compared Hy-SimulMT and the human experts in terms of latency and translation quality. For the human experts, the highest score among the three selected individuals was used. In this capability benchmarking assessment, Hy-SimulMT achieved translation quality comparable to human simultaneous interpretation experts (with higher scores) and faster speed (with lower latency).
Reference