ui.Chat

ui.Chat(self, id, *, messages=(), on_error='auto', tokenizer=None)

Create a chat interface.

A UI component for building conversational interfaces. With it, end users can submit messages, which will cause a .on_user_submit() callback to run. That callback gets passed the user input message, which can be used to generate a response. The response can then be appended to the chat using .append_message() or .append_message_stream().

Here's a rough outline for how to implement a Chat:

from shiny.express import ui

# Create and display chat instance
chat = ui.Chat(id="my_chat")
chat.ui()

# Define a callback to run when the user submits a message
@chat.on_user_submit
async def handle_user_input(user_input: str):
    # Create a response message stream
    response = await my_model.generate_response(user_input, stream=True)
    # Append the response into the chat
    await chat.append_message_stream(response)

In the outline above, my_model.generate_response() is a placeholder for the function that generates a response based on the chat's messages. This function will look different depending on the model you're using, but it will generally involve passing the messages to the model and getting a response back. Also, you'll typically have a choice to stream=True the response generation, and in that case, you'll use .append_message_stream() instead of .append_message() to append the response to the chat. Streaming is preferrable when available since it allows for more responsive and scalable chat interfaces.

It is also highly recommended to use a package like chatlas to generate responses, especially when responses should be aware of the chat history, support tool calls, etc. See this article to learn more.

Parameters

id : str

A unique identifier for the chat session. In Shiny Core, make sure this id matches a corresponding chat_ui call in the UI.

messages : Sequence[Any] = ()

A sequence of messages to display in the chat. A given message can be one of the following: * A string, which is interpreted as markdown and rendered to HTML on the client. * To prevent interpreting as markdown, mark the string as HTML. * A UI element (specifically, a TagChild). * This includes TagList, which take UI elements (including strings) as children. In this case, strings are still interpreted as markdown as long as they’re not inside HTML. * A dictionary with content and role keys. The content key can contain a content as described above, and the role key can be “assistant” or “user”. NOTE: content may include specially formatted input suggestion links (see .append_message() for more information).

on_error : Literal[‘auto’, ‘actual’, ‘sanitize’, ‘unhandled’] = 'auto'

How to handle errors that occur in response to user input. When "unhandled", the app will stop running when an error occurs. Otherwise, a notification is displayed to the user and the app continues to run. * "auto": Sanitize the error message if the app is set to sanitize errors, otherwise display the actual error message. * "actual": Display the actual error message to the user. * "sanitize": Sanitize the error message before displaying it to the user. * "unhandled": Do not display any error message to the user.

tokenizer : TokenEncoding | None = None

The tokenizer to use for calculating token counts, which is required to impose token_limits in .messages(). If not provided, a default generic tokenizer is attempted to be loaded from the tokenizers library. A specific tokenizer may also be provided by following the TokenEncoding (tiktoken or tozenizers) protocol (e.g., tiktoken.encoding_for_model("gpt-4o")).

Examples

#| standalone: true
#| components: [editor, viewer]
#| layout: vertical
#| viewerHeight: 400

## file: app.py
from shiny import App, ui

app_ui = ui.page_fillable(
    ui.panel_title("Hello Shiny Chat"),
    ui.chat_ui("chat"),
    fillable_mobile=True,
)

# Create a welcome message
welcome = """
Hi! This is a simple Shiny `Chat` UI. Enter a message below and I will
simply repeat it back to you.

To learn more about chatbots and how to build them with Shiny, check out
[the documentation](https://shiny.posit.co/py/docs/genai-chatbots.html).
"""


def server(input, output, session):
    chat = ui.Chat(id="chat", messages=[welcome])

    # Define a callback to run when the user submits a message
    @chat.on_user_submit
    async def handle_user_input(user_input: str):
        # Append a response to the chat
        await chat.append_message(f"You said: {user_input}")


app = App(app_ui, server)


## file: requirements.txt
shiny


## file: _template.json
{
  "type": "app",
  "id": "chat-hello",
  "title": "Hello Shiny Chat",
  "next_steps": [
    "Run the app with `shiny run app.py`."
  ]
}

Attributes

Name Description
latest_message_stream React to changes in the latest message stream.

Methods

Name Description
append_message Append a message to the chat.
append_message_stream Append a message as a stream of message chunks.
clear_messages Clear all chat messages.
destroy Destroy the chat instance.
enable_bookmarking Enable bookmarking for the chat instance.
message_stream_context Message stream context manager.
messages Reactively read chat messages
on_user_submit Define a function to invoke when user input is submitted.
set_user_message Deprecated. Use update_user_input(value=value) instead.
transform_assistant_response Transform assistant responses.
transform_user_input Transform user input.
update_user_input Update the user input.
user_input Reactively read the user’s message.

append_message

ui.Chat.append_message(message, *, icon=None)

Append a message to the chat.

Parameters

message : Any

A given message can be one of the following: * A string, which is interpreted as markdown and rendered to HTML on the client. * To prevent interpreting as markdown, mark the string as HTML. * A UI element (specifically, a TagChild). * This includes TagList, which take UI elements (including strings) as children. In this case, strings are still interpreted as markdown as long as they’re not inside HTML. * A dictionary with content and role keys. The content key can contain content as described above, and the role key can be “assistant” or “user”. NOTE: content may include specially formatted input suggestion links (see note below).

icon : HTML | Tag | TagList | None = None

An optional icon to display next to the message, currently only used for assistant messages. The icon can be any HTML element (e.g., an img tag) or a string of HTML.

Note

Input suggestions

Input suggestions are special links that send text to the user input box when clicked (or accessed via keyboard). They can be created in the following ways:

  • <span class='suggestion'>Suggestion text</span>: An inline text link that places ‘Suggestion text’ in the user input box when clicked.
  • <img data-suggestion='Suggestion text' src='image.jpg'>: An image link with the same functionality as above.
  • <span data-suggestion='Suggestion text'>Actual text</span>: An inline text link that places ‘Suggestion text’ in the user input box when clicked.

A suggestion can also be submitted automatically by doing one of the following:

  • Adding a submit CSS class or a data-suggestion-submit="true" attribute to the suggestion element.
  • Holding the Ctrl/Cmd key while clicking the suggestion link.

Note that a user may also opt-out of submitting a suggestion by holding the Alt/Option key while clicking the suggestion link.

Streamed messages

Use .append_message_stream() instead of this method when stream=True (or similar) is specified in model’s completion method.

append_message_stream

ui.Chat.append_message_stream(message, *, icon=None)

Append a message as a stream of message chunks.

Parameters

message : Iterable[Any] | AsyncIterable[Any]

An (async) iterable of message chunks. Each chunk can be one of the following: * A string, which is interpreted as markdown and rendered to HTML on the client. * To prevent interpreting as markdown, mark the string as HTML. * A UI element (specifically, a TagChild). * This includes TagList, which take UI elements (including strings) as children. In this case, strings are still interpreted as markdown as long as they’re not inside HTML. * A dictionary with content and role keys. The content key can contain content as described above, and the role key can be “assistant” or “user”. NOTE: content may include specially formatted input suggestion links (see note below).

icon : HTML | Tag | None = None

An optional icon to display next to the message, currently only used for assistant messages. The icon can be any HTML element (e.g., an img tag) or a string of HTML.

Note

Input suggestions are special links that send text to the user input box when
clicked (or accessed via keyboard). They can be created in the following ways:

* `<span class='suggestion'>Suggestion text</span>`: An inline text link that
    places 'Suggestion text' in the user input box when clicked.
* `<img data-suggestion='Suggestion text' src='image.jpg'>`: An image link with
    the same functionality as above.
* `<span data-suggestion='Suggestion text'>Actual text</span>`: An inline text
    link that places 'Suggestion text' in the user input box when clicked.

A suggestion can also be submitted automatically by doing one of the following:

* Adding a `submit` CSS class or a `data-suggestion-submit="true"` attribute to
  the suggestion element.
* Holding the `Ctrl/Cmd` key while clicking the suggestion link.

Note that a user may also opt-out of submitting a suggestion by holding the
`Alt/Option` key while clicking the suggestion link.
Use this method (over `.append_message()`) when `stream=True` (or similar) is
specified in model's completion method.

Returns

:

An extended task that represents the streaming task. The .result() method of the task can be called in a reactive context to get the final state of the stream.

clear_messages

ui.Chat.clear_messages()

Clear all chat messages.

destroy

ui.Chat.destroy()

Destroy the chat instance.

enable_bookmarking

ui.Chat.enable_bookmarking(client, /, *, bookmark_on='response')

Enable bookmarking for the chat instance.

This method registers on_bookmark and on_restore hooks on session.bookmark (shiny.bookmark.Bookmark) to save/restore chat state on both the Chat and client= instances. In order for this method to actually work correctly, a bookmark_store= must be specified in shiny.App().

Parameters

client : ClientWithState | chatlas.Chat[Any, Any]

The chat client instance to use for bookmarking. This can be a Chat model provider from chatlas, or more generally, an instance following the ClientWithState protocol.

bookmark_on : Optional[Literal[‘response’]] = 'response'

The event to trigger the bookmarking on. Supported values include: - "response" (the default): a bookmark is triggered when the assistant is done responding. - None: no bookmark is triggered When this method triggers a bookmark, it also updates the URL query string to reflect the bookmarked state.

Raises

: ValueError

If the Shiny App does have bookmarking enabled.

Returns

: CancelCallback

A callback to cancel the bookmarking hooks.

message_stream_context

ui.Chat.message_stream_context()
    Message stream context manager.

    A context manager for appending streaming messages into the chat. This context
    manager can:

    1. Be used in isolation to append a new streaming message to the chat.
        * Compared to `.append_message_stream()` this method is more flexible but
          isn&#x27;t non-blocking by default (i.e., it doesn&#x27;t launch an extended task).
    2. Be nested within itself
        * Nesting is primarily useful for making checkpoints to `.clear()` back
          to (see the example below).
    3. Be used from within a `.append_message_stream()`
        * Useful for inserting additional content from another context into the
          stream (e.g., see the note about tool calls below).

Yields

    :
        A `MessageStream` class instance, which has a method for `.append()`ing
        message content chunks to as well as way to `.clear()` the stream back to
        it's initial state. Note that `.append()` supports the same message content
        types as `.append_message()`.

Example

    ```python
    import asyncio

    from shiny import reactive
    from shiny.express import ui

    chat = ui.Chat(id="my_chat")
    chat.ui()

    @reactive.effect
    async def _():
        async with chat.message_stream_context() as msg:
            await msg.append("Starting stream...

Progress:“) async with chat.message_stream_context() as progress: for x in [0, 50, 100]: await progress.append(f” {x}%“) await asyncio.sleep(1) await progress.clear() await msg.clear() await msg.append(”Completed stream”) ```

Note

    A useful pattern for displaying tool calls in a chatbot is for the tool to
    display using `.message_stream_context()` while the the response generation is
    happening through `.append_message_stream()`. This allows the tool to display
    things like progress updates (or other "ephemeral" content) and optionally
    `.clear()` the stream back to it's initial state when ready to display the
    "final" content.

messages

ui.Chat.messages(
    format=MISSING,
    token_limits=None,
    transform_user='all',
    transform_assistant=False,
)

Reactively read chat messages

Obtain chat messages within a reactive context. The default behavior is intended for passing messages along to a model for response generation where you typically want to:

  1. Cap the number of tokens sent in a single request (i.e., token_limits).
  2. Apply user input transformations (i.e., transform_user), if any.
  3. Not apply assistant response transformations (i.e., transform_assistant) since these are predominantly for display purposes (i.e., the model shouldn't concern itself with how the responses are displayed).

Parameters

format : MISSING_TYPE | ProviderMessageFormat = MISSING

The message format to return. The default value of MISSING means chat messages are returned as ChatMessage objects (a dictionary with content and role keys). Other supported formats include: * "anthropic": Anthropic message format. * "google": Google message (aka content) format. * "langchain": LangChain message format. * "openai": OpenAI message format. * "ollama": Ollama message format.

token_limits : tuple[int, int] | None = None

Limit the conversation history based on token limits. If specified, only the most recent messages that fit within the token limits are returned. This is useful for avoiding “exceeded token limit” errors when sending messages to the relevant model, while still providing the most recent context available. A specified value must be a tuple of two integers. The first integer is the maximum number of tokens that can be sent to the model in a single request. The second integer is the amount of tokens to reserve for the model’s response. Note that token counts based on the tokenizer provided to the Chat constructor.

transform_user : Literal[‘all’, ‘last’, ‘none’] = 'all'

Whether to return user input messages with transformation applied. This only matters if a transform_user_input was provided to the chat constructor. The default value of "all" means all user input messages are transformed. The value of "last" means only the last user input message is transformed. The value of "none" means no user input messages are transformed.

transform_assistant : bool = False

Whether to return assistant messages with transformation applied. This only matters if an transform_assistant_response was provided to the chat constructor.

Note

Messages are listed in the order they were added. As a result, when this method is called in a .on_user_submit() callback (as it most often is), the last message will be the most recent one submitted by the user.

Returns

: tuple[ChatMessage, …]

A tuple of chat messages.

on_user_submit

ui.Chat.on_user_submit(fn=None)

Define a function to invoke when user input is submitted.

Apply this method as a decorator to a function (fn) that should be invoked when the user submits a message. This function can take an optional argument, which will be the user input message.

In many cases, the implementation of fn should also do the following:

  1. Generate a response based on the user input.
  • If the response should be aware of chat history, use a package like chatlas to manage the chat state, or use the .messages() method to get the chat history.
  1. Append that response to the chat component using .append_message() ( or .append_message_stream() if the response is streamed).

Parameters

fn : UserSubmitFunction | None = None

A function to invoke when user input is submitted.

Note

This method creates a reactive effect that only gets invalidated when the user submits a message. Thus, the function fn can read other reactive dependencies, but it will only be re-invoked when the user submits a message.

set_user_message

ui.Chat.set_user_message(value)

Deprecated. Use update_user_input(value=value) instead.

transform_assistant_response

ui.Chat.transform_assistant_response(fn=None)

Transform assistant responses.

Use this method as a decorator on a function (fn) that transforms assistant responses before displaying them in the chat. This is useful for post-processing model responses before displaying them to the user.

Parameters

fn : TransformAssistantResponseFunction | None = None

A function that takes a string and returns either a string, shiny.ui.HTML, or None. If fn returns a string, it gets interpreted and parsed as a markdown on the client (and the resulting HTML is then sanitized). If fn returns shiny.ui.HTML, it will be displayed as-is. If fn returns None, the response is effectively ignored.

Note

When doing an .append_message_stream(), fn gets called on every chunk of the response (thus, it should be performant), and can optionally access more information (i.e., arguments) about the stream. The 1st argument (required) contains the accumulated content, the 2nd argument (optional) contains the current chunk, and the 3rd argument (optional) is a boolean indicating whether this chunk is the last one in the stream.

transform_user_input

ui.Chat.transform_user_input(fn=None)

Transform user input.

Use this method as a decorator on a function (fn) that transforms user input before storing it in the chat messages returned by .messages(). This is useful for implementing RAG workflows, like taking a URL and scraping it for text before sending it to the model.

Parameters

fn : TransformUserInput | TransformUserInputAsync | None = None

A function to transform user input before storing it in the chat .messages(). If fn returns None, the user input is effectively ignored, and .on_user_submit() callbacks are suspended until more input is submitted. This behavior is often useful to catch and handle errors that occur during transformation. In this case, the transform function should append an error message to the chat (via .append_message()) to inform the user of the error.

update_user_input

ui.Chat.update_user_input(
    value=None,
    placeholder=None,
    submit=False,
    focus=False,
)

Update the user input.

Parameters

value : str | None = None

The value to set the user input to.

placeholder : str | None = None

The placeholder text for the user input.

submit : bool = False

Whether to automatically submit the text for the user. Requires value.

focus : bool = False

Whether to move focus to the input element. Requires value.

user_input

ui.Chat.user_input(transform=False)

Reactively read the user's message.

Parameters

transform : bool = False

Whether to apply the user input transformation function (if one was provided).

Returns

: str | None

The user input message (before any transformation).

Note

Most users shouldn’t need to use this method directly since the last item in .messages() contains the most recent user input. It can be useful for:

  1. Taking a reactive dependency on the user’s input outside of a .on_user_submit() callback.
  2. Maintaining message state separately from .messages().