Skip to main content

embedding.EmbeddingNVIDIA

Creates an embedding function provider backed by NVIDIA’s NIM embedding models.

Usage

Source

embedding.EmbeddingNVIDIA(
    model="nvidia/llama-nemotron-embed-1b-v2",
    base_url="https://integrate.api.nvidia.com/v1",
    api_key=None,
    batch_size=20,
    truncate="NONE",
)

Implements the EmbeddingProvider interface.

NVIDIA’s embedding API is compatible with the OpenAI client library. It supports differentiated query vs document embeddings via the input_type parameter.

Browse available models and get API keys at NVIDIA NIM.

NVIDIA also provides Docker images for self-hosting NIM models. When running a model locally or on your own infrastructure, set the base_url parameter to point to your self-hosted endpoint (e.g., "http://localhost:8000/v1").

Parameters

model: str = "nvidia/llama-nemotron-embed-1b-v2"

The NVIDIA embedding model to use. Default is “nvidia/llama-nemotron-embed-1b-v2”.

base_url: str = "https://integrate.api.nvidia.com/v1"

The base URL for the NVIDIA API. Default is “https://integrate.api.nvidia.com/v1”. Change this to point to a self-hosted NIM container (e.g., "http://localhost:8000/v1").

api_key: str | None = None

The API key for authenticating with NVIDIA. If None, it will use the NVIDIA_API_KEY environment variable if set. Not required when using a self-hosted NIM container.

batch_size: int = 20

The number of texts to process in each batch when calling the API.

truncate: str = "NONE"
Truncation strategy. Default is “NONE”. Set to “END” to truncate inputs that exceed the model’s max token length.

Examples

from raghilda.embedding import EmbeddingNVIDIA, EmbedInputType

# Using NVIDIA's hosted API
provider = EmbeddingNVIDIA(model="nvidia/llama-nemotron-embed-1b-v2")

# Or using a self-hosted NIM container
provider = EmbeddingNVIDIA(
    model="nvidia/llama-nemotron-embed-1b-v2",
    base_url="http://localhost:8000/v1",
)

# Embed documents for indexing
doc_embeddings = provider.embed(
    ["Hello world", "Testing embeddings"],
    input_type=EmbedInputType.DOCUMENT,
)

# Embed a query for search
query_embedding = provider.embed(
    ["How do I test embeddings?"],
    input_type=EmbedInputType.QUERY,
)