OpenTelemetry
Observability and telemetry instrumentation for Shiny applications
otel
otel
OpenTelemetry instrumentation for Shiny applications.
OpenTelemetry support for observing Shiny application behavior, performance, and reactive execution.
Quick Start Example
from shiny import App, ui, render, reactive
from shiny import otel
app_ui = ui.page_fluid(
ui.input_slider("n", "N", 1, 100, 50),
ui.output_text("result"),
ui.output_text("result_private"),
ui.output_text("result_instrumented"),
)
def server(input, output, session):
@render.text
def result():
# Full Shiny telemetry for this output
return f"Value: {input.n()}"
@render.text
@otel.suppress # Disables Shiny's internal telemetry for sensitive operations
def result_private():
return f"Private value: {input.n()}"
@render.text
@otel.collect # Enables Shiny's internal telemetry even when default is suppressed
def result_instrumented():
return f"Instrumented value: {input.n()}"
app = App(app_ui, server)Run under OpenTelemetry zero-code auto-instrumentation (installed with pip install "shiny[otel]"):
opentelemetry-instrument --traces_exporter console shiny run app.pyWatch the console output to see Shiny's spans for result and result_instrumented but not for result_private. Note that the app contains no OpenTelemetry setup code — instrumentation is applied at launch.
Table of Contents
What is OpenTelemetry?
OpenTelemetry is an open-source observability framework that provides a standardized way to collect telemetry data (traces, metrics, and logs) from applications. It's vendor-neutral and widely supported by observability platforms.
Key concepts:
- Traces: Records of requests flowing through your application, showing timing and dependencies
- Spans: Individual units of work within a trace (e.g., a function execution)
- Logs: Structured log events with context
- Attributes: Key-value metadata attached to spans and logs
Why Use OpenTelemetry with Shiny?
Shiny applications have complex reactive execution flows that can be difficult to debug and optimize. OpenTelemetry provides:
1. Reactive Flow Visualization
See exactly how reactive computations propagate through your app: - Which calcs and effects execute during each update cycle - Parent-child relationships between reactive components - Execution timing and ordering
2. Performance Analysis
Identify bottlenecks in your application: - Which outputs take the longest to render - Which reactive computations are slow - How many reactive invalidations occur per user interaction
3. Debugging Aid
Understand unexpected behavior: - Why certain reactive computations run (or don't run) - Execution order when multiple things invalidate - Async operation context propagation
4. Production Monitoring
Track application health in production: - Session lifecycle and user behavior patterns - Error rates and types - Performance over time
Getting Started
Installation
Install Shiny with OpenTelemetry support:
pip install "shiny[otel]"This installs the OpenTelemetry API (required), the SDK (for exporters), and opentelemetry-distro[otlp] (for zero-code auto-instrumentation and OTLP export).
Standard Setup: opentelemetry-instrument
The standard way to enable OpenTelemetry is the opentelemetry-instrument wrapper, which configures the SDK before your app starts — your app contains no instrumentation code at all:
OTEL_SERVICE_NAME=my-shiny-app opentelemetry-instrument shiny run app.pyBy default this exports traces over OTLP to http://localhost:4317. Everything is configurable through standard OpenTelemetry environment variables or CLI flags. For example, to print spans to the console while developing:
opentelemetry-instrument --traces_exporter console shiny run app.pyNotes:
- Set
OTEL_SERVICE_NAME(orOTEL_RESOURCE_ATTRIBUTES); otherwise traces are reported withservice.name: unknown_service. - Auto-instrumentation also instruments other libraries your app uses (HTTP clients, databases, etc.), so their spans appear alongside Shiny's.
- Works with
shiny run --reload— the reloaded process inherits the instrumentation.
Quick Example
from shiny import App, ui, render
app_ui = ui.page_fluid(
ui.input_slider("n", "N", 1, 100, 50),
ui.output_text("result"),
)
def server(input, output, session):
@render.text
def result():
return f"Value: {input.n()}"
app = App(app_ui, server)Run with:
opentelemetry-instrument --traces_exporter console shiny run app.pyYou'll see OpenTelemetry spans printed to the console showing Shiny's internal execution. See examples/open-telemetry/ for a complete app demonstrating collection control.
Code-based Setup (Discouraged)
Configuring OpenTelemetry inside the app is discouraged: it couples your app to a specific observability setup, and it conflicts with external instrumentation. Providers can only be installed once per process — under opentelemetry-instrument the wrapper's provider is installed before your app code runs, so a manual trace.set_tracer_provider() call logs Overriding of current TracerProvider is not allowed and is silently ignored. Configure OpenTelemetry in exactly one place.
Legitimate reasons to configure in code include observability SDKs that manage OpenTelemetry themselves (e.g. logfire.configure() — see Observability Backends) and deployment platforms where you cannot control the launch command. In those cases, guard the setup so it only runs when nothing else has configured a provider yet:
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor
provider = trace.get_tracer_provider()
already_configured = isinstance(provider, TracerProvider) or isinstance(
getattr(provider, "provider", None), TracerProvider
)
if not already_configured:
new_provider = TracerProvider()
new_provider.add_span_processor(SimpleSpanProcessor(ConsoleSpanExporter()))
trace.set_tracer_provider(new_provider)Note: Shiny uses lazy initialization for its OpenTelemetry tracer, so it picks up whichever provider is installed by the time the app runs, regardless of setup method.
Collection Levels
Shiny provides five collection levels to control the granularity of telemetry:
none
No Shiny telemetry collected. Use when you want to completely disable Shiny's instrumentation while keeping your own custom spans.
Overhead: None Use case: Disabling telemetry entirely
session
Only session lifecycle spans (session start/end, HTTP/WebSocket connections).
Overhead: Minimal (1-2 spans per session) Use case: Basic session tracking in production
reactive_update
Session spans + reactive update cycle spans (one span per flush cycle).
Overhead: Low (1 span per reactive flush) Use case: Understanding how many update cycles occur
reactivity
Everything from reactive_update + individual reactive execution spans (calcs, effects, outputs, extended tasks) + value update logs.
Overhead: Moderate (1 span per reactive computation) Use case: Detailed debugging and development
all
All available telemetry (currently equivalent to reactivity). Reserved for future expansion.
Overhead: Moderate Use case: Maximum observability
Setting Collection Level
Via environment variable:
SHINY_OTEL_COLLECT=session \
opentelemetry-instrument shiny run app.py
# or: none, reactive_update, reactivity, allThe default level is all if not specified.
Configuration
Environment Variables
SHINY_OTEL_COLLECT
Sets the default collection level for the application.
# Minimal overhead - session lifecycle only
export SHINY_OTEL_COLLECT=session
# Balanced - update cycles tracked
export SHINY_OTEL_COLLECT=reactive_update
# Full detail - all reactive executions
export SHINY_OTEL_COLLECT=reactivity
# Maximum (same as reactivity currently)
export SHINY_OTEL_COLLECT=allOpenTelemetry SDK Configuration
The OpenTelemetry SDK itself supports many configuration options via environment variables:
# Service name
export OTEL_SERVICE_NAME=my-shiny-app
# OTLP exporter endpoint
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
# Resource attributes
export OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production,service.version=1.0.0
# Trace sampling
export OTEL_TRACES_SAMPLER=parentbased_traceidratio
export OTEL_TRACES_SAMPLER_ARG=0.1 # Sample 10% of tracesSee OpenTelemetry SDK Configuration for full details.
Programmatic Control
Important: The suppression setting for a reactive object (calc, effect, output) is captured at initialization time – when the reactive object is created – not when the reactive function executes. This means otel.suppress affects whether telemetry is suppressed on the reactive object during its definition, and that setting is used for all subsequent executions.
Precedence: otel.suppress and otel.collect are absolute per-object settings that take precedence over the global SHINY_OTEL_COLLECT level. suppress forces telemetry off for the stamped objects even when the global level is all, and collect forces it on (level ALL) even when the global level is lower (e.g. session) – so a collect-stamped output still produces spans in a SHINY_OTEL_COLLECT=session run. Infrastructure spans (session_start, session_end, reactive_update) follow only the environment variable and are never affected by suppress/collect.
Decorator
Use otel.suppress as a decorator to disable Shiny telemetry for a reactive function. The decorator stamps the suppression setting on the function, and the reactive object reads it when it is created:
from shiny import otel
@reactive.calc
@otel.suppress
def sensitive_computation():
"""This entire calc runs without Shiny telemetry on every execution."""
api_key = input.api_key()
return validate_api_key(api_key)Important: When decorating reactive objects, apply otel.suppress before (i.e., closer to the function than) the reactive decorator:
# Correct order -- otel.suppress is applied to the function first,
# then @reactive.calc reads the stamped setting at initialization time
@reactive.calc
@otel.suppress
def my_calc():
pass
# Incorrect - will raise TypeError
@otel.suppress # Cannot wrap a reactive object
@reactive.calc
def my_calc():
passContext Manager (Initialization Time Only)
Use otel.suppress() as a context manager to suppress telemetry during reactive object creation. Any reactive objects defined inside the with block will have telemetry suppressed:
from shiny import otel
with otel.suppress():
# Reactive objects created here are never instrumented
@reactive.calc
def sensitive_calc():
return load_secrets()
# Reactive objects created outside use the default level
@reactive.calc
def normal_calc():
return load_public_data()Does NOT work at runtime: Using with otel.suppress() inside a reactive function body has no effect on Shiny's internal telemetry for that reactive object, because the suppression setting was already captured when the object was created:
@reactive.calc
def load_secrets():
... # This part is instrumented with Shiny telemetry
@reactive.calc
def my_calc():
# THIS DOES NOT WORK as intended for Shiny telemetry.
# `load_secrets()` will still generate spans/logs because
# reactive objects are captured at initialization time.
with otel.suppress():
sensitive_data = load_secrets()
return sensitive_dataotel.collect Decorator
Use otel.collect as a decorator to enable Shiny's internal telemetry for a reactive function when the default level is suppressed:
from shiny import otel
@reactive.calc
@otel.collect
def instrumented_computation():
"""This calc always runs with Shiny telemetry, regardless of context."""
return load_public_data()otel.collect Context Manager (Initialization Time Only)
Use otel.collect() as a context manager to enable telemetry during reactive object creation. Any reactive objects defined inside the with block will have telemetry enabled:
from shiny import otel
with otel.suppress():
# Reactive objects created here have telemetry suppressed
with otel.collect():
@reactive.calc
def public_calc():
# This calc has telemetry enabled despite the outer suppress
return load_public_data()
@reactive.calc
def private_calc():
# Back to suppressed
return load_private_data()Best Practices
1. Use Batch Processing in Production
opentelemetry-instrument uses batching span processors by default — nothing to do. If you configure the SDK in code instead, replace SimpleSpanProcessor with BatchSpanProcessor, which reduces overhead by buffering spans and sending them in batches:
from opentelemetry.sdk.trace.export import BatchSpanProcessor
provider.add_span_processor(BatchSpanProcessor(exporter))2. Choose Appropriate Collection Level
Development:
export SHINY_OTEL_COLLECT=reactivity # Full detail for debuggingProduction:
export SHINY_OTEL_COLLECT=session # Minimal overhead
# or
export SHINY_OTEL_COLLECT=reactive_update # Balanced3. Add Resource Attributes
Include service metadata in your traces via standard environment variables:
export OTEL_SERVICE_NAME=my-shiny-app
export OTEL_RESOURCE_ATTRIBUTES="service.version=1.2.3,deployment.environment=production,service.namespace=analytics-team"
opentelemetry-instrument shiny run app.py4. Protect Sensitive Data
Use otel.suppress for operations involving sensitive data:
from shiny import otel
@reactive.calc
@otel.suppress
def process_credentials():
"""Disable telemetry for credential handling."""
username = input.username()
password = input.password()
return authenticate(username, password)Remember: otel.suppress only disables Shiny's internal telemetry. Your own custom OpenTelemetry spans are unaffected.
5. Enable Error Sanitization
When app.sanitize_errors=True, Shiny automatically sanitizes error messages in spans to prevent leaking sensitive information:
app = App(app_ui, server, sanitize_errors=True)6. Use Sampling in High-Traffic Apps
For high-traffic applications, use trace sampling to reduce overhead. Via standard environment variables:
# Sample 10% of traces
export OTEL_TRACES_SAMPLER=parentbased_traceidratio
export OTEL_TRACES_SAMPLER_ARG=0.1
opentelemetry-instrument shiny run app.py7. Add Custom Spans for Business Logic
Complement Shiny's spans with your own for business-critical operations:
from opentelemetry import trace
tracer = trace.get_tracer(__name__)
@reactive.calc
def expensive_computation():
with tracer.start_as_current_span("database_query") as span:
span.set_attribute("query.type", "analytics")
result = run_query()
span.set_attribute("query.rows", len(result))
return resultObservability Backends
Shiny's OpenTelemetry integration works with any OTLP-compatible backend. In every case the app itself stays unchanged — pick the backend by setting standard OTEL_* environment variables and launching with opentelemetry-instrument.
Jaeger (Open Source)
Perfect for local development and self-hosted monitoring.
Setup:
docker run -d --name jaeger \
-p 16686:16686 \
-p 4317:4317 \
jaegertracing/all-in-one:latestConfiguration:
# OTLP to http://localhost:4317 is the default, so only the service name is needed
OTEL_SERVICE_NAME=my-shiny-app opentelemetry-instrument shiny run app.pyOpen the Jaeger UI to explore your Shiny app's traces. You'll see: - Session lifecycle spans - Reactive update cycles - Individual calc/effect/output executions - Timing and nesting relationships
Pydantic Logfire (Managed)
Modern observability platform with excellent Python support.
Zero-code configuration (recommended): Logfire accepts OTLP directly, so the standard opentelemetry-instrument setup works with no app-code changes. Create a write token in your Logfire project settings, then:
export OTEL_SERVICE_NAME=my-shiny-app
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf # Logfire speaks OTLP over HTTP, not gRPC
export OTEL_EXPORTER_OTLP_ENDPOINT="https://logfire-us.pydantic.dev" # or logfire-eu
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=$LOGFIRE_TOKEN"
opentelemetry-instrument shiny run app.pyAlternative — Logfire SDK (code-based): the logfire package configures OpenTelemetry itself and adds auto-detected credentials (logfire auth) and its own instrumentation helpers, at the cost of touching app code:
pip install logfireimport logfire
# Configure BEFORE importing Shiny
logfire.configure(
token=os.environ["LOGFIRE_TOKEN"],
service_name="my-shiny-app",
)
# Now import Shiny
from shiny import App, uiNote: use exactly one of the two — logfire.configure() installs the OpenTelemetry provider itself, so with the SDK route run the app directly (shiny run app.py) and do not also wrap it with opentelemetry-instrument.
Honeycomb (Managed)
Powerful observability platform focused on trace analysis.
UI: ui.honeycomb.io
Configuration:
export OTEL_SERVICE_NAME=my-shiny-app
export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.honeycomb.io"
export OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=$HONEYCOMB_API_KEY"
opentelemetry-instrument shiny run app.pyDatadog (Managed)
Enterprise observability platform with APM features.
Configuration: send OTLP to your Datadog Agent (with OTLP ingestion enabled):
export OTEL_SERVICE_NAME=my-shiny-app
export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317"
opentelemetry-instrument shiny run app.pyNew Relic (Managed)
Full-stack observability platform.
UI: one.newrelic.com
Configuration:
export OTEL_SERVICE_NAME=my-shiny-app
export OTEL_EXPORTER_OTLP_ENDPOINT="https://otlp.nr-data.net:4317"
export OTEL_EXPORTER_OTLP_HEADERS="api-key=$NEW_RELIC_LICENSE_KEY"
opentelemetry-instrument shiny run app.pyConsole (Development)
Simple console output for debugging. See the open-telemetry example for a complete working demonstration of console output with collection control.
opentelemetry-instrument --traces_exporter console shiny run app.pyTroubleshooting
No Spans Appearing
Problem: Console/backend shows no spans from Shiny.
Solutions:
Verify the app is launched with
opentelemetry-instrument(or, for code-based setup, that a provider is installed before the app runs)Check collection level:
# Make sure it's not "none" export SHINY_OTEL_COLLECT=allVerify exporter endpoint is correct and reachable
Check for error messages in console
Too Much Overhead
Problem: Application performance degraded with OpenTelemetry enabled.
Solutions:
Lower collection level:
export SHINY_OTEL_COLLECT=session # MinimalEnable sampling:
export OTEL_TRACES_SAMPLER=parentbased_traceidratio export OTEL_TRACES_SAMPLER_ARG=0.1 # Sample 10% of tracesIf configuring the SDK in code, use
BatchSpanProcessorinstead ofSimpleSpanProcessor(opentelemetry-instrumentalready batches by default):from opentelemetry.sdk.trace.export import BatchSpanProcessor provider.add_span_processor(BatchSpanProcessor(exporter))Use
@otel.suppressfor high-frequency operations:@reactive.calc @otel.suppress def high_frequency_calc(): pass
Sensitive Data in Traces
Problem: Passwords or API keys appearing in span attributes.
Solutions:
Use
@otel.suppressdecorator on sensitive reactive functions:@reactive.calc @otel.suppress def process_api_keys(): api_key = input.api_key() return validate(api_key)Use
with otel.suppress():when defining reactive objects that handle sensitive data (the setting is captured at initialization time):with otel.suppress(): @reactive.calc def handle_password(): password = input.password() return hash_password(password)Enable error sanitization:
app = App(app_ui, server, sanitize_errors=True)Use
otel.collectto re-enable telemetry for specific calcs inside a broad suppress block:with otel.suppress(): # Most of the app has telemetry suppressed with otel.collect(): @reactive.calc def public_calc(): return load_public_data()
Spans Not Nested Correctly
Problem: Parent-child relationships incorrect in traces.
Solutions:
Ensure you're using async context propagation correctly
Check that custom spans use
start_as_current_span():# CORRECT with tracer.start_as_current_span("my_span"): pass # WRONG - breaks context chain span = tracer.start_span("my_span")
Backend Not Receiving Traces
Problem: OpenTelemetry configured but backend shows no data.
Solutions:
Check exporter endpoint URL and authentication
Verify network connectivity to backend
Check backend-specific requirements (headers, format)
Use the console exporter first to verify spans are generated:
opentelemetry-instrument --traces_exporter console shiny run app.py
"Overriding of current TracerProvider is not allowed"
Problem: This warning appears at startup, and your code-based exporter configuration seems to have no effect.
Cause: The app calls trace.set_tracer_provider() while also running under opentelemetry-instrument, which already installed a tracer provider before your app code ran. The manual call is ignored.
Solution: Pick one configuration method:
- Keep
opentelemetry-instrumentand delete the manual setup — configure exporters viaOTEL_*environment variables or CLI flags instead, or - Keep the manual setup and run the app directly (
shiny run app.py).
ImportError: No module named 'opentelemetry'
Problem: OpenTelemetry not installed.
Solution:
pip install "shiny[otel]"Next Steps
- Examples: Check out
examples/open-telemetry/for working example apps - API Reference: See API documentation for
shiny.otelmodule - OpenTelemetry Docs: opentelemetry.io/docs/languages/python
- Shiny Docs: shiny.posit.co/py
Getting Help
- GitHub Issues: github.com/posit-dev/py-shiny/issues
- Community: forum.posit.co/c/shiny
- OpenTelemetry Community: cloud-native.slack.com (#otel-python channel)
otel.suppress
otel.suppress(func=None)Disable Shiny's internal OTel instrumentation for a function or block.
Serves a dual purpose depending on how it is called:
- As a no-parens decorator (
@otel.suppress): Stamps the plain function withOtelCollectLevel.NONEat definition time. Reactive objects created from the function will not emit Shiny internal spans or logs. - As a context manager (
with otel.suppress():): Sets the collection level toNONEfor the duration of the block. Reactive objects created inside the block captureNONEas their level.
Parameters
func : Any = None-
The plain function to suppress. Only provided when used as a decorator (
@otel.suppress, no parens). Must be a plain callable — passing areactive.calc,reactive.effect, or renderer object raisesTypeErrorwith instructions for the correct decorator ordering.
Returns
: Any-
When used as a decorator: the original function, unchanged except for the
_shiny_otel_collect_levelattribute being set. When used as a context manager: an_OtelContextinstance whose__exit__restores the previous level viaContextVar.reset.
Raises
: TypeError-
If applied to a
reactive.calc,reactive.effect, or renderer object (@otel.suppressmust come before those decorators), or to any non-callable.
Note
Only affects spans and logs created by Shiny itself (reactive calculations and value updates). User-defined OpenTelemetry spans are unaffected.
Collection level is captured at initialization time for reactive objects — when reactive.calc, reactive.effect, or reactive.value is instantiated. Changing the context variable after initialization has no effect on already-created reactive objects.
Both otel.suppress and otel.collect are backed by a ContextVar and are async-safe: concurrent tasks each see their own level independently.
Examples
Decorator (no parens):
from shiny import reactive, otel
@reactive.calc
@otel.suppress
def sensitive_calc():
return load_api_key()Context manager (parens required):
from shiny import reactive, otel
with otel.suppress():
private_counter = reactive.value(0)
@reactive.calc
def private_calc():
return private_counter() * 2Nested with otel.collect to re-enable for one object:
from shiny import reactive, otel
with otel.suppress():
@reactive.calc
def private_calc(): # suppressed
return load_private_data()
with otel.collect():
@reactive.calc
def public_calc(): # re-enabled
return load_public_data()See Also
otel.collect
otel.collect(func=None)Enable Shiny's internal OTel instrumentation for a function or block.
Counterpart to suppress. Useful when the global default has been lowered via SHINY_OTEL_COLLECT or when inside a with otel.suppress(): block and a specific reactive object needs telemetry re-enabled.
Serves a dual purpose depending on how it is called:
- As a no-parens decorator (
@otel.collect): Stamps the plain function withOtelCollectLevel.ALLat definition time. Reactive objects created from the function will emit Shiny internal spans and logs regardless of the surrounding context. - As a context manager (
with otel.collect():): Sets the collection level toALLfor the duration of the block. Reactive objects created inside the block captureALLas their level.
Parameters
func : Any = None-
The plain function to enable collection for. Only provided when used as a decorator (
@otel.collect, no parens). Must be a plain callable — passing areactive.calc,reactive.effect, or renderer object raisesTypeErrorwith instructions for the correct decorator ordering.
Returns
: Any-
When used as a decorator: the original function, unchanged except for the
_shiny_otel_collect_levelattribute being set. When used as a context manager: an_OtelContextinstance whose__exit__restores the previous level viaContextVar.reset.
Raises
: TypeError-
If applied to a
reactive.calc,reactive.effect, or renderer object (@otel.collectmust come before those decorators), or to any non-callable.
Note
Only affects spans and logs created by Shiny itself. User-defined OpenTelemetry spans are unaffected.
Collection level is captured at initialization time for reactive objects. otel.collect overrides the surrounding context level — including a SHINY_OTEL_COLLECT=none environment variable — for reactive objects created within its scope.
Both otel.collect and otel.suppress are backed by a ContextVar and are async-safe: concurrent tasks each see their own level independently.
Examples
Decorator (no parens) — override a low global default:
from shiny import reactive, otel
# Even with SHINY_OTEL_COLLECT=none, this calc is always instrumented
@reactive.calc
@otel.collect
def public_calc():
return load_public_data()Context manager — re-enable within a suppress block:
from shiny import reactive, otel
with otel.suppress():
@reactive.calc
def private_calc():
return load_private_data() # suppressed
with otel.collect():
@reactive.calc
def public_calc():
return load_public_data() # re-enabledSee Also
otel.get_level
otel.get_level()Get the current OpenTelemetry collect level.
The collect level is determined in the following order:
- Context variable (set via
otel.suppress()context manager) - SHINY_OTEL_COLLECT environment variable
- Default: ALL
Returns
:OtelCollectLevel-
The current collect level.
Examples
Check the current collection level:
from shiny import otel
# Get the current level
level = otel.get_level()
print(f"Current level: {level.name}") # e.g., "ALL", "SESSION", "NONE"Use with suppress context manager:
from shiny import otel
print(otel.get_level().name) # "ALL" (default)
with otel.suppress():
print(otel.get_level().name) # "NONE"
print(otel.get_level().name) # "ALL" (restored)