Models

Under the hood, querychat is powered by chatlas, a library for building chat-based applications with large language models (LLMs). chatlas supports a wide range of LLM providers – see here for a full list.

Specify a model

To use a particular model, pass a "{provider}/{model}" string to the client parameter, which gets passed along to chatlas.ChatAuto

from querychat import QueryChat
from querychat.data import titanic

qc = QueryChat(
    titanic(),
    "titanic",
    client="anthropic/claude-sonnet-4-5"
)

And, if you’d like to effectively set a new default model, you can use the QUERYCHAT_CLIENT environment variable.

export QUERYCHAT_CLIENT="anthropic/claude-sonnet-4-5"

Note that it can also be useful to pass a full Chat object to the client parameter for more advanced use cases (e.g., custom parameters, tools, etc). It can also be useful for getting some helpful autocomplete of available models.

from chatlas import ChatAnthropic

client = ChatAnthropic(model="claude-sonnet-4-5")

Credentials

Most models require an API key or some other form of authentication. See the reference page for the relevant model provider (e.g., ChatAnthropic) to learn more on how to set up credentials.

Free trial options

If you don’t want to spend any money to get started, ChatPosit provides free trial access to a curated set of models through the Posit AI gateway.

posit-model.py

from chatlas import ChatPosit

# Prompts you to log in via OAuth the first time; tokens are then cached
client = ChatPosit()

In general, most providers will prefer credentials stored as environment variables, and common practice is to use a .env file to manage these variables. For example, for ChatOpenAI(), you might create a .env file like so:

.env

OPENAI_API_KEY="your_api_key_here"

Then, load the environment variables via the dotenv package:

pip install dotenv

from dotenv import load_dotenv
load_dotenv()

Recommended models

In theory, you could use any model that has tool calling support, but we currently recommend (as of November 2025):

• GPT-4.1 (the default)
• Claude 4.5 Sonnet
• Google Gemini 3.0

In our testing, we’ve found that those models strike a good balance between accuracy and latency. That said, smaller/faster models like GPT-4.1-mini or Claude Haiku 4.5 work well for most tables and are worth trying first—they’re significantly cheaper and faster. You can always switch to a larger model if you find the results aren’t meeting your needs. On the other end of the spectrum, reasoning models like o3-mini tend to slow down responses without providing meaningfully better results for this task.

We’ve also seen some decent results with frontier local models (e.g., gpt-oss:20b), but even if you have the compute to run the largest models, they still tend to lag behind the cloud-hosted options in terms of accuracy and speed.

Data privacy concerns?

If you have data privacy concerns, consider that your org may provide access to private instances of these models with data residency guarantees. For example, Azure, AWS Bedrock, and Google Vertex AI all provide private instances of popular LLMs. You can interface with these enterprise providers by passing the right string (e.g., "bedrock-anthropic") or Chat object (e.g., ChatBedrockAnthropic()) to the client parameter. See the chatlas docs for more details.