NVIDIA AI Releases Nemotron 3 Embed: An Open Embedding Collection Whose 8B Checkpoint Ranks #1 on RTEB

Embedding fashions resolve which passages an agent ever sees. NVIDIA launched Nemotron 3 Embed model to work on that layer. It targets production-scale RAG, agentic retrieval, code retrieval, and agent reminiscence.

What is Nemotron 3 Embed?

The mannequin assortment contains three open checkpoints. Nemotron-3-Embed-8B-BF16 is the accuracy-first possibility. Nemotron-3-Embed-1B-BF16 carries the identical design right into a smaller footprint. Nemotron-3-Embed-1B-NVFP4 is the Blackwell-optimized 4-bit path.

All three are transformer encoders skilled with bidirectional consideration masking. The ultimate embedding comes from common pooling over token-level representations. Maximum sequence size is 32,768 tokens on each checkpoint.

Each mannequin was evaluated throughout 34 languages. All three carry the OpenMDW License Agreement, model 1.1 (OpenMDW-1.1). Notably, the bases are Mistral fashions. The 8B is constructed with Ministral-3-8B-Instruct-2512. Both 1B variants use Ministral-3-3B-Instruct-2512.

Performance

Nemotron-3-Embed-8B-BF16 ranks #1 total on RTEB (as of July 17 2026), the Retrieval Embedding Benchmark. Evaluation covers its 16 public duties. Every determine beneath is common NDCG@10, at mannequin sequence size 4096.

Two gaps are price noting. The 1B good points 10.4 RTEB factors over llama-nemotron-embed-vl-1b-v2, the prior-generation baseline. Separately, NVFP4 prices 0.38 RTEB factors towards its BF16 mum or dad, or 99.5% retention.

How the 1B Model was Built?

Those 1B scores come from a compression pipeline, not a smaller coaching run. The mum or dad was nemotron-3-embed-3b, pruned and distilled throughout two iterative rounds.

First, the 3B mum or dad was pruned to 2B utilizing NVIDIA ModelDecide mcore_minitron Neural Architecture Search (NAS). The search covers hidden width, FFN measurement, consideration heads, and depth. It then picks the most effective candidate from the top-10 Pareto entrance. A 50k in-domain calibration corpus scored these candidates.

Next, the 2B mannequin was distilled from the fine-tuned 8B embedding trainer. Distillation mixed cosine distance loss (COS) and imply squared error (MSE) loss. The knowledge mix was multilingual and in-domain. Finally, the identical process repeated to provide the 1.14B checkpoint.

The NVFP4 Serving Tradeoff

Compression then continues into the serving format. Quantization hit weights and activations of linear layers solely, concentrating on the NVFP4 knowledge sort. The analysis workforce used nvidia-modelopt v0.45.0. Quantization-Aware Distillation (QAD) adopted, primarily to get well accuracy on lengthy inputs.

Calibration used 512 samples: 256 queries and 256 passages from abisee/cnn_dailymail. QAD coaching used 20k samples.

The rsesearch workforce stories NVFP4 on Blackwell delivers as much as 2x increased throughput than BF16. It retains 99%+ of BF16 retrieval accuracy. The NVFP4 card additionally paperwork dynamic embedding sizes. You can slice the 2048-d vector from the begin to 1024 or 512 dimensions. Re-normalize afterward.

Interactive Explainer: The Five-Stage Retrieval Path

Before touching code, watch the trail run. It animates prefixing, bidirectional encoding, common pooling, L2 normalization, and dot-product scoring. Scores come from every card’s revealed anticipated output.

(*3*)

Deployment Matrix

As that walkthrough implies, the checkpoints don’t share runtime paths.

Alongside the checkpoints, NVIDIA analysis workforce launched an optimized NIM microservice for the 1B mannequin. The Rust-based NIM matches or outperforms the vLLM checkpoint on GB200 and RTX PRO 6000. NVIDIA examined enter sequence lengths of 256 and 1024. Separately, NVIDIA NeMo AutoModel recipes cowl fine-tuning and distillation.

Using It in Code

With these paths in thoughts, prefixes come first. Queries take question: and paperwork take passage: . Embeddings are L2-normalized, so dot product equals cosine similarity.

# pip set up --upgrade "transformers>=5.2.0" "sentence-transformers>=5.4.1"
import torch
from sentence_transformers import SentenceTransformer

QUERIES = ["How can someone reduce exposure to pollen during allergy season?"]
DOCUMENTS = ["People with pollen allergy can reduce exposure by staying indoors "
             "on dry, windy days, avoiding early-morning outdoor activity, and "
             "going outside after rain when pollen levels are lower."]

mannequin = SentenceTransformer(
    "nvidia/Nemotron-3-Embed-8B-BF16",
    system="cuda",
    model_kwargs={"dtype": torch.bfloat16,
                  # use "sdpa" if FlashAttention-2 is unavailable
                  "attn_implementation": "flash_attention_2"},
    processor_kwargs={"padding_side": "left"},
)
mannequin.max_seq_length = 32768

q = mannequin.encode_query(QUERIES, batch_size=1, convert_to_tensor=True)
d = mannequin.encode_document(DOCUMENTS, batch_size=1, convert_to_tensor=True)
print(mannequin.similarity(q, d))  # card's revealed q[3]/d[3] rating: 0.8008

encode_query and encode_document learn the saved prompts. So you by no means add prefixes by hand. For serving, /v2/embed applies them from input_type as a substitute:

vllm serve nvidia/Nemotron-3-Embed-1B-NVFP4 
  --max-model-len 4096 
  --max-num-batched-tokens 4096 
  --max-cudagraph-capture-size 4096

import numpy as np, requests

def embed(input_type: str, texts: listing[str]) -> np.ndarray:
    r = requests.publish(
        "http://localhost:8000/v2/embed",
        json={"mannequin": "nvidia/Nemotron-3-Embed-1B-NVFP4",
              "input_type": input_type,          # "question" or "doc"
              "texts": texts,
              "embedding_types": ["float"],
              "truncate": "END"},
        timeout=120,
    )
    r.raise_for_status()
    return np.array(r.json()["embeddings"]["float"], dtype=np.float32)

scores = embed("question", QUERIES) @ embed("doc", DOCUMENTS).T

Use Cases With Examples

• Multilingual enterprise search: A assist workforce indexes Hindi, Japanese, and English tickets collectively. Because retrieval is cross-lingual, a German question can floor a Japanese decision observe.
• Code retrieval: Training included coir_apps, coir_cosqa, synthetic_text2sql, and SWE-bench. Natural-language-to-code lookup is subsequently nearer to in-distribution.
• Agent reminiscence: The 32,768-token restrict lets an agent embed lengthy dialog summaries with out aggressive chunking.
• Cost-tiered RAG: Serve 1B-NVFP4 for high-volume recall, and route onerous queries to the 8B. Because widths differ, this wants two indexes.

Key Takeaways

• Nemotron-3-Embed-8B-BF16 ranks #1 on RTEB at 78.46 avg NDCG@10.
• Three open checkpoints span 8B BF16, 1B BF16, and 1B NVFP4.
• NVFP4 retains 99%+ of BF16 accuracy at as much as 2x Blackwell throughput.
• The 1B got here from ModelDecide NAS pruning plus COS+MSE distillation from the 8B.
• All checkpoints use OpenMDW-1.1 and assist 32,768-token inputs.

Check out the NVIDIA launch post on Hugging Face, Nemotron 3 Embed collection, 8B-BF16 card, 1B-BF16 card and 1B-NVFP4 card. Also, be happy to comply with us on Twitter and don’t neglect to affix our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Need to associate with us for selling your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar and so on.? Connect with us

The publish NVIDIA AI Releases Nemotron 3 Embed: An Open Embedding Collection Whose 8B Checkpoint Ranks #1 on RTEB appeared first on MarkTechPost.