Termsslug: tokenization

Tokenization

词元化

The process of converting text sequences into numerical IDs for model processing

Curated Intelligence

3 items · Page 1/1
腾讯技术工程 (微信公众号)T2·Models
78 pts
Tencent Unveils TGR: A Generative Paradigm for Industrial Recommendation Systems

Tencent introduces the TGR (Tencent Generative Recommendation) stack to address structural bottlenecks in traditional cascaded recommendation systems, such as lack of scaling laws and poor cold-start performance. By leveraging GenRank (CCFormer), GenRec (end-to-end list generation), and Reason (LLM-enhanced inference), TGR has been deployed across 10+ scenarios with significant metric improvements. The work, published in top conferences like AAAI and ACL, marks a pivotal shift from discriminative to generative paradigms in industrial recommendations.

Hacker News AIT2·Tools
58 pts
Open-Source Guide Building LLMs from Scratch in Node.js and TypeScript

This project offers a comprehensive guide to building Large Language Models from scratch using Node.js and TypeScript, covering data loaders, embeddings, attention mechanisms, and Transformer blocks. All code is packaged as runnable modules on GitHub, aiming to demystify model internals by reimplementing core concepts in a non-Python environment.