Meet ARGUS: A Scalable AI Framework for Training Large Recommender Transformers to One Billion Parameters

Yandex has launched ARGUS (AutoRegressive Generative User Sequential modeling), a large-scale transformer-based framework for recommender methods that scales up to one billion parameters. This breakthrough locations Yandex amongst a small group of worldwide expertise leaders — alongside Google, Netflix, and Meta — which have efficiently overcome the long-standing technical limitations in scaling recommender transformers.

Breaking Technical Barriers in Recommender Systems

Recommender methods have lengthy struggled with three cussed constraints: short-term reminiscence, restricted scalability, and poor adaptability to shifting person conduct. Conventional architectures trim person histories down to a small window of current interactions, discarding months or years of behavioral information. The result’s a shallow view of intent that misses long-term habits, delicate shifts in style, and seasonal cycles. As catalogs broaden into the billions of things, these truncated fashions not solely lose precision but in addition choke on the computational calls for of personalization at scale. The consequence is acquainted: stale suggestions, decrease engagement, and fewer alternatives for serendipitous discovery.

Very few corporations have efficiently scaled recommender transformers past experimental setups. Google, Netflix, and Meta have invested closely on this space, reporting features from architectures like YouTubeDNN, PinnerFormer, and Meta’s Generative Recommenders. With ARGUS, Yandex joins this choose group of corporations demonstrating billion-parameter recommender fashions in reside companies. By modeling complete behavioral timelines, the system uncovers each apparent and hidden correlations in person exercise. This long-horizon perspective permits ARGUS to seize evolving intent and cyclical patterns with far larger constancy. For instance, as an alternative of reacting solely to a current buy, the mannequin learns to anticipate seasonal behaviors—like routinely surfacing the popular model of tennis balls when summer time approaches—with out requiring the person to repeat the identical indicators yr after yr.

Technical Innovations Behind ARGUS

The framework introduces a number of key advances:

• Dual-objective pre-training: ARGUS decomposes autoregressive studying into two subtasks — next-item prediction and suggestions prediction. This mixture improves each imitation of historic system conduct and modeling of true person preferences.
• Scalable transformer encoders: Models scale from 3.2M to 1B parameters, with constant efficiency enhancements throughout all metrics. At the billion-parameter scale, pairwise accuracy uplift elevated by 2.66%, demonstrating the emergence of a scaling regulation for recommender transformers.
• Extended context modeling: ARGUS handles person histories up to 8,192 interactions lengthy in a single go, enabling personalization over months of conduct fairly than simply the previous few clicks.
• Efficient fine-tuning: A two-tower structure permits offline computation of embeddings and scalable deployment, decreasing inference price relative to prior target-aware or impression-level on-line fashions.

Real-World Deployment and Measured Gains

ARGUS has already been deployed at scale on Yandex’s music platform, serving tens of millions of customers. In manufacturing A/B assessments, the system achieved:

• +2.26% improve in complete listening time (TLT)
  
• +6.37% improve in like chance

These represent the biggest recorded high quality enhancements within the platform’s historical past for any deep learning–based mostly recommender mannequin.

Future Directions

Yandex researchers plan to lengthen ARGUS to real-time advice duties, discover characteristic engineering for pairwise rating, and adapt the framework to high-cardinality domains akin to giant e-commerce and video platforms. The demonstrated capacity to scale user-sequence modeling with transformer architectures means that recommender methods are poised to comply with a scaling trajectory related to pure language processing.

Conclusion

With ARGUS, Yandex has established itself as one of many few world leaders driving state-of-the-art recommender methods. By brazenly sharing its breakthroughs, the corporate isn’t solely bettering personalization throughout its personal companies but in addition accelerating the evolution of advice applied sciences for your complete trade.

---

Check out the PAPER here. Thanks to the Yandex workforce for the thought management/ Resources for this text.

The publish Meet ARGUS: A Scalable AI Framework for Training Large Recommender Transformers to One Billion Parameters appeared first on MarkTechPost.