Audio Language Model

Automatic speech recognition (ASR) is becoming a core building block for AI products, from meeting tools to voice agents. Mistral’s new Voxtral Transcribe 2 family targets this space with 2 models that split cleanly into batch and realtime use cases, while keeping cost, latency, and deployment constraints in focus. The release includes: Voxtral Mini Transcribe…

Meta researchers have introduced Perception Encoder Audiovisual, PEAV, as a new family of encoders for joint audio and video understanding. The model learns aligned audio, video, and text representations in a single embedding space using large scale contrastive training on about 100M audio video pairs with text captions. From Perception Encoder to PEAV Perception Encoder,…

Why do present audio AI fashions usually carry out worse after they generate longer reasoning as a substitute of grounding their selections within the precise sound. StepEnjoyable analysis group releases Step-Audio-R1, a brand new audio LLM designed for check time compute scaling, tackle this failure mode by displaying that the accuracy drop with chain of…

How do you construct a single speech recognition system that may perceive 1,000’s of languages together with many who by no means had working ASR (computerized speech recognition) fashions earlier than? Meta AI has launched Omnilingual ASR, an open supply speech recognition suite that scales to greater than 1,600 languages and could be prolonged to…

Can a speech enhancer educated solely on actual noisy recordings cleanly separate speech and noise—with out ever seeing paired information? A crew of researchers from Brno University of Technology and Johns Hopkins University proposes Unsupervised Speech Enhancement utilizing Data-defined Priors (USE-DDP), a dual-stream encoder–decoder that separates any noisy enter into two waveforms—estimated clear speech and…

Liquid AI has launched LFM2-Audio-1.5B, a compact audio–language basis mannequin that each understands and generates speech and textual content via a single end-to-end stack. It positions itself for low-latency, real-time assistants on resource-constrained units, extending the LFM2 household into audio whereas retaining a small footprint. https://www.liquid.ai/weblog/lfm2-audio-an-end-to-end-audio-foundation-model But what’s really new? a unified spine with disentangled…

Xiaomi’s MiMo crew launched MiMo-Audio, a 7-billion-parameter audio-language mannequin that runs a single next-token goal over interleaved textual content and discretized speech, scaling pretraining past 100 million hours of audio. What’s really new? Instead of relying on task-specific heads or lossy acoustic tokens, MiMo-Audio makes use of a bespoke RVQ (residual vector quantization) tokenizer that…