Chat

Chat(provider, turns=None)

A chat object that can be used to interact with a language model.

A Chat is an sequence of sequence of user and assistant Turns sent to a specific Provider. A Chat takes care of managing the state associated with the chat; i.e. it records the messages that you send to the server, and the messages that you receive back. If you register a tool (i.e. an function that the assistant can call on your behalf), it also takes care of the tool loop.

You should generally not create this object yourself, but instead call ChatOpenAI or friends instead.

Attributes

Name	Description
current_display	Get the currently active markdown display, if any.
system_prompt	A property to get (or set) the system prompt for the chat.

Methods

Name	Description
app	Enter a web-based chat app to interact with the LLM.
chat	Generate a response from the chat.
chat_async	Generate a response from the chat asynchronously.
console	Enter a chat console to interact with the LLM.
export	Export the chat history to a file.
extract_data	Extract structured data from the given input.
extract_data_async	Extract structured data from the given input asynchronously.
get_last_turn	Get the last turn in the chat with a specific role.
get_turns	Get all the turns (i.e., message contents) in the chat.
on_tool_request	Register a callback for a tool request event.
on_tool_result	Register a callback for a tool result event.
register_tool	Register a tool (function) with the chat.
set_echo_options	Set echo styling options for the chat.
set_turns	Set the turns of the chat.
stream	Generate a response from the chat in a streaming fashion.
stream_async	Generate a response from the chat in a streaming fashion asynchronously.
token_count	Get an estimated token count for the given input.
token_count_async	Get an estimated token count for the given input asynchronously.
tokens	Get the tokens for each turn in the chat.

app

Chat.app(
    stream=True,
    port=0,
    launch_browser=True,
    bg_thread=None,
    echo=None,
    content='all',
    kwargs=None,
)

Enter a web-based chat app to interact with the LLM.

Parameters

Name	Type	Description	Default
stream	bool	Whether to stream the response (i.e., have the response appear in chunks).	True
port	int	The port to run the app on (the default is 0, which will choose a random port).	0
launch_browser	bool	Whether to launch a browser window.	True
bg_thread	Optional[bool]	Whether to run the app in a background thread. If None, the app will run in a background thread if the current environment is a notebook.	None
echo	Optional[EchoOptions]	One of the following (defaults to "none" when stream=True and "text" when stream=False): - "text": Echo just the text content of the response. - "output": Echo text and tool call content. - "all": Echo both the assistant and user turn. - "none": Do not echo any content.	None
content	Literal['text', 'all']	Whether to display text content or all content (i.e., tool calls).	'all'
kwargs	Optional[SubmitInputArgsT]	Additional keyword arguments to pass to the method used for requesting the response.	None

chat

Chat.chat(*args, echo='output', stream=True, kwargs=None)

Generate a response from the chat.

Parameters

Name	Type	Description	Default
args	Content | str	The user input(s) to generate a response from.	()
echo	EchoOptions	One of the following (default is “output”): - "text": Echo just the text content of the response. - "output": Echo text and tool call content. - "all": Echo both the assistant and user turn. - "none": Do not echo any content.	'output'
stream	bool	Whether to stream the response (i.e., have the response appear in chunks).	True
kwargs	Optional[SubmitInputArgsT]	Additional keyword arguments to pass to the method used for requesting the response.	None

Returns

Name	Type	Description
	ChatResponse	A (consumed) response from the chat. Apply str() to this object to get the text content of the response.

chat_async

Chat.chat_async(*args, echo='output', stream=True, kwargs=None)

Generate a response from the chat asynchronously.

Parameters

Name	Type	Description	Default
args	Content | str	The user input(s) to generate a response from.	()
echo	EchoOptions	One of the following (default is “output”): - "text": Echo just the text content of the response. - "output": Echo text and tool call content. - "all": Echo both the assistant and user turn. - "none": Do not echo any content.	'output'
stream	bool	Whether to stream the response (i.e., have the response appear in chunks).	True
kwargs	Optional[SubmitInputArgsT]	Additional keyword arguments to pass to the method used for requesting the response.	None

Returns

Name	Type	Description
	ChatResponseAsync	A (consumed) response from the chat. Apply str() to this object to get the text content of the response.

console

Chat.console(echo='output', stream=True, kwargs=None)

Enter a chat console to interact with the LLM.

To quit, input ‘exit’ or press Ctrl+C.

Parameters

Name	Type	Description	Default
echo	EchoOptions	One of the following (default is “output”): - "text": Echo just the text content of the response. - "output": Echo text and tool call content. - "all": Echo both the assistant and user turn. - "none": Do not echo any content.	'output'
stream	bool	Whether to stream the response (i.e., have the response appear in chunks).	True
kwargs	Optional[SubmitInputArgsT]	Additional keyword arguments to pass to the method used for requesting the response	None

Returns

Name	Type	Description
	None

export

Chat.export(
    filename,
    *,
    turns=None,
    title=None,
    content='text',
    include_system_prompt=True,
    overwrite=False,
)

Export the chat history to a file.

Parameters

Name	Type	Description	Default
filename	str | Path	The filename to export the chat to. Currently this must be a .md or .html file.	required
turns	Optional[Sequence[Turn]]	The .get_turns() to export. If not provided, the chat’s current turns will be used.	None
title	Optional[str]	A title to place at the top of the exported file.	None
overwrite	bool	Whether to overwrite the file if it already exists.	False
content	Literal['text', 'all']	Whether to include text content, all content (i.e., tool calls), or no content.	'text'
include_system_prompt	bool	Whether to include the system prompt in a	True

Returns

Name	Type	Description
	Path	The path to the exported file.

extract_data

Chat.extract_data(*args, data_model, echo='none', stream=False)

Extract structured data from the given input.

Parameters

Name	Type	Description	Default
args	Content | str	The input to extract data from.	()
data_model	type[BaseModel]	A Pydantic model describing the structure of the data to extract.	required
echo	EchoOptions	One of the following (default is “none”): - "text": Echo just the text content of the response. - "output": Echo text and tool call content. - "all": Echo both the assistant and user turn. - "none": Do not echo any content.	'none'
stream	bool	Whether to stream the response (i.e., have the response appear in chunks).	False

Returns

Name	Type	Description
	dict[str, Any]	The extracted data.

extract_data_async

Chat.extract_data_async(*args, data_model, echo='none', stream=False)

Extract structured data from the given input asynchronously.

Parameters

Name	Type	Description	Default
args	Content | str	The input to extract data from.	()
data_model	type[BaseModel]	A Pydantic model describing the structure of the data to extract.	required
echo	EchoOptions	One of the following (default is “none”): - "text": Echo just the text content of the response. - "output": Echo text and tool call content. - "all": Echo both the assistant and user turn. - "none": Do not echo any content.	'none'
stream	bool	Whether to stream the response (i.e., have the response appear in chunks). Defaults to True if echo is not “none”.	False

Returns

Name	Type	Description
	dict[str, Any]	The extracted data.

get_last_turn

Chat.get_last_turn(role='assistant')

Get the last turn in the chat with a specific role.

Parameters

Name	Type	Description	Default
role	Literal['assistant', 'user', 'system']	The role of the turn to return.	'assistant'

get_turns

Chat.get_turns(include_system_prompt=False)

Get all the turns (i.e., message contents) in the chat.

Parameters

Name	Type	Description	Default
include_system_prompt	bool	Whether to include the system prompt in the turns.	False

on_tool_request

Chat.on_tool_request(callback)

Register a callback for a tool request event.

A tool request event occurs when the assistant requests a tool to be called on its behalf. Before invoking the tool, on_tool_request handlers are called with the relevant ContentToolRequest object. This is useful if you want to handle tool requests in a custom way, such as requiring logging them or requiring user approval before invoking the tool

Parameters

Name	Type	Description	Default
callback	Callable[[ContentToolRequest], None]	A function to be called when a tool request event occurs. This function must have a single argument, which will be the tool request (i.e., a ContentToolRequest object).	required

Returns

Name	Type	Description
	A callable that can be used to remove the callback later.

on_tool_result

Chat.on_tool_result(callback)

Register a callback for a tool result event.

A tool result event occurs when a tool has been invoked and the result is ready to be provided to the assistant. After the tool has been invoked, on_tool_result handlers are called with the relevant ContentToolResult object. This is useful if you want to handle tool results in a custom way such as logging them.

Parameters

Name	Type	Description	Default
callback	Callable[[ContentToolResult], None]	A function to be called when a tool result event occurs. This function must have a single argument, which will be the tool result (i.e., a ContentToolResult object).	required

Returns

Name	Type	Description
	A callable that can be used to remove the callback later.

register_tool

Chat.register_tool(func, *, model=None)

Register a tool (function) with the chat.

The function will always be invoked in the current Python process.

Examples

If your tool has straightforward input parameters, you can just register the function directly (type hints and a docstring explaning both what the function does and what the parameters are for is strongly recommended):

from chatlas import ChatOpenAI, Tool


def add(a: int, b: int) -> int:
    '''
    Add two numbers together.

####     Parameters {.doc-section .doc-section-----parameters}

    a : int
        The first number to add.
    b : int
        The second number to add.
    '''
    return a + b


chat = ChatOpenAI()
chat.register_tool(add)
chat.chat("What is 2 + 2?")

If your tool has more complex input parameters, you can provide a Pydantic model that corresponds to the input parameters for the function, This way, you can have fields that hold other model(s) (for more complex input parameters), and also more directly document the input parameters:

from chatlas import ChatOpenAI, Tool
from pydantic import BaseModel, Field


class AddParams(BaseModel):
    '''Add two numbers together.'''

    a: int = Field(description="The first number to add.")

    b: int = Field(description="The second number to add.")


def add(a: int, b: int) -> int:
    return a + b


chat = ChatOpenAI()
chat.register_tool(add, model=AddParams)
chat.chat("What is 2 + 2?")

Parameters

func The function to be invoked when the tool is called. model A Pydantic model that describes the input parameters for the function. If not provided, the model will be inferred from the function’s type hints. The primary reason why you might want to provide a model in Note that the name and docstring of the model takes precedence over the name and docstring of the function.

set_echo_options

Chat.set_echo_options(rich_markdown=None, rich_console=None, css_styles=None)

Set echo styling options for the chat.

Parameters

Name	Type	Description	Default
rich_markdown	Optional[dict[str, Any]]	A dictionary of options to pass to rich.markdown.Markdown(). This is only relevant when outputting to the console.	None
rich_console	Optional[dict[str, Any]]	A dictionary of options to pass to rich.console.Console(). This is only relevant when outputting to the console.	None
css_styles	Optional[dict[str, str]]	A dictionary of CSS styles to apply to IPython.display.Markdown(). This is only relevant when outputing to the browser.	None

set_turns

Chat.set_turns(turns)

Set the turns of the chat.

This method is primarily useful for clearing or setting the turns of the chat (i.e., limiting the context window).

Parameters

Name	Type	Description	Default
turns	Sequence[Turn]	The turns to set. Turns with the role “system” are not allowed.	required

stream

Chat.stream(*args, content='text', echo='none', kwargs=None)

Generate a response from the chat in a streaming fashion.

Parameters

Name	Type	Description	Default
args	Content | str	The user input(s) to generate a response from.	()
content	Literal['text', 'all']	Whether to yield just text content or include rich content objects (e.g., tool calls) when relevant.	'text'
echo	EchoOptions	One of the following (default is “none”): - "text": Echo just the text content of the response. - "output": Echo text and tool call content. - "all": Echo both the assistant and user turn. - "none": Do not echo any content.	'none'
kwargs	Optional[SubmitInputArgsT]	Additional keyword arguments to pass to the method used for requesting the response.	None

Returns

Name	Type	Description
	ChatResponse	An (unconsumed) response from the chat. Iterate over this object to consume the response.

stream_async

Chat.stream_async(*args, content='text', echo='none', kwargs=None)

Generate a response from the chat in a streaming fashion asynchronously.

Parameters

Name	Type	Description	Default
args	Content | str	The user input(s) to generate a response from.	()
content	Literal['text', 'all']	Whether to yield just text content or include rich content objects (e.g., tool calls) when relevant.	'text'
echo	EchoOptions	One of the following (default is “none”): - "text": Echo just the text content of the response. - "output": Echo text and tool call content. - "all": Echo both the assistant and user turn. - "none": Do not echo any content.	'none'
kwargs	Optional[SubmitInputArgsT]	Additional keyword arguments to pass to the method used for requesting the response.	None

Returns

Name	Type	Description
	ChatResponseAsync	An (unconsumed) response from the chat. Iterate over this object to consume the response.

token_count

Chat.token_count(*args, data_model=None)

Get an estimated token count for the given input.

Estimate the token size of input content. This can help determine whether input(s) and/or conversation history (i.e., .get_turns()) should be reduced in size before sending it to the model.

Parameters

Name	Type	Description	Default
args	Content | str	The input to get a token count for.	()
data_model	Optional[type[BaseModel]]	If the input is meant for data extraction (i.e., .extract_data()), then this should be the Pydantic model that describes the structure of the data to extract.	None

Returns

Name	Type	Description
	int	The token count for the input.

Note

Remember that the token count is an estimate. Also, models based on ChatOpenAI() currently does not take tools into account when estimating token counts.

Examples

from chatlas import ChatAnthropic

chat = ChatAnthropic()
# Estimate the token count before sending the input
print(chat.token_count("What is 2 + 2?"))

# Once input is sent, you can get the actual input and output
# token counts from the chat object
chat.chat("What is 2 + 2?", echo="none")
print(chat.token_usage())

token_count_async

Chat.token_count_async(*args, data_model=None)

Get an estimated token count for the given input asynchronously.

Estimate the token size of input content. This can help determine whether input(s) and/or conversation history (i.e., .get_turns()) should be reduced in size before sending it to the model.

Parameters

Name	Type	Description	Default
args	Content | str	The input to get a token count for.	()
data_model	Optional[type[BaseModel]]	If this input is meant for data extraction (i.e., .extract_data_async()), then this should be the Pydantic model that describes the structure of the data to extract.	None

Returns

Name	Type	Description
	int	The token count for the input.

tokens

Chat.tokens(values='discrete')

Get the tokens for each turn in the chat.

Parameters

Name	Type	Description	Default
values	Literal['cumulative', 'discrete']	If “cumulative” (the default), the result can be summed to get the chat’s overall token usage (helpful for computing overall cost of the chat). If “discrete”, the result can be summed to get the number of tokens the turns will cost to generate the next response (helpful for estimating cost of the next response, or for determining if you are about to exceed the token limit).	'discrete'

Returns

Name	Type	Description
	list[int]	A list of token counts for each (non-system) turn in the chat. The 1st turn includes the tokens count for the system prompt (if any).

Raises

Name	Type	Description
	ValueError	If the chat’s turns (i.e., .get_turns()) are not in an expected format. This may happen if the chat history is manually set (i.e., .set_turns()). In this case, you can inspect the “raw” token values via the .get_turns() method (each turn has a .tokens attribute).