ChatHuggingFace

ChatHuggingFace(system_prompt=None, model=None, api_key=None, kwargs=None)

Chat with a model hosted on Hugging Face Inference API.

Hugging Face hosts a variety of open-source and proprietary AI models available via their Inference API. To use the Hugging Face API, you must have an Access Token, which you can obtain from your Hugging Face account. Ensure that at least “Make calls to Inference Providers” and “Make calls to your Inference Endpoints” is checked.

Prerequisites

API key

You will need to create a Hugging Face account and generate an API token from your account settings. Make sure to enable “Make calls to Inference Providers” permission.

Examples

import os
from chatlas import ChatHuggingFace

chat = ChatHuggingFace(api_key=os.getenv("HUGGINGFACE_API_KEY"))
chat.chat("What is the capital of France?")

Parameters

Name	Type	Description	Default
system_prompt	Optional[str]	A system prompt to set the behavior of the assistant.	None
model	Optional[str]	The model to use for the chat. The default, None, will pick a reasonable default, and warn you about it. We strongly recommend explicitly choosing a model for all but the most casual use.	None
api_key	Optional[str]	The API key to use for authentication. You generally should not supply this directly, but instead set the HUGGINGFACE_API_KEY environment variable.	None
kwargs	Optional['ChatClientArgs']	Additional arguments to pass to the underlying OpenAI client constructor.	None

Returns

Name	Type	Description
	Chat	A chat object that retains the state of the conversation.

Known limitations

• Some models do not support the chat interface or parts of it, for example google/gemma-2-2b-it does not support a system prompt. You will need to carefully choose the model.
• Tool calling support varies by model - many models do not support it.

Note

This function is a lightweight wrapper around ChatOpenAI, with the defaults tweaked for Hugging Face.

Note

Pasting an API key into a chat constructor (e.g., ChatHuggingFace(api_key="...")) is the simplest way to get started, and is fine for interactive use, but is problematic for code that may be shared with others.

Instead, consider using environment variables or a configuration file to manage your credentials. One popular way to manage credentials is to use a .env file to store your credentials, and then use the python-dotenv package to load them into your environment.

pip install python-dotenv

# .env
HUGGINGFACE_API_KEY=...

from chatlas import ChatHuggingFace
from dotenv import load_dotenv

load_dotenv()
chat = ChatHuggingFace()
chat.console()

Another, more general, solution is to load your environment variables into the shell before starting Python (maybe in a .bashrc, .zshrc, etc. file):

export HUGGINGFACE_API_KEY=...