Getting Started

Prerequisites

Python 3.10+
uv (recommended) or pip
just (optional, for running common tasks)

Installation

Clone the repository and install dependencies:

git clone https://github.com/posit-dev/vip.git
cd vip

# Set up with uv (recommended)
uv sync
uv run playwright install chromium

# Or set up with pip
pip install -e ".[dev]"
playwright install chromium

Configuration

Copy the example config and edit it for your deployment:

cp vip.toml.example vip.toml

Each product section (Connect, Workbench, Package Manager) can be enabled or disabled individually. Secrets should be set via environment variables:

Variable	Purpose
`VIP_CONNECT_API_KEY`	Connect admin API key
`VIP_WORKBENCH_API_KEY`	Workbench admin API key
`VIP_PM_TOKEN`	Package Manager token
`VIP_TEST_USERNAME`	Test user login name
`VIP_TEST_PASSWORD`	Test user login password

You can also point to the config file explicitly:

pytest --vip-config=/path/to/vip.toml

Authentication

Password / LDAP / Keycloak

Set credentials via environment variables and run normally:

export VIP_TEST_USERNAME="test-user"
export VIP_TEST_PASSWORD="test-password"
uv run pytest

Okta / external OIDC provider

External identity providers require an interactive browser login. Use --interactive-auth to launch a visible browser, authenticate through the IdP, then run the remaining tests headlessly:

uv run pytest --interactive-auth

This opens a Chromium window, waits for you to complete the OIDC flow, captures the session, and runs all tests headlessly with the authenticated state.

Running tests

Run the full suite or target specific products:

# Run all tests
uv run pytest

# Run tests for a single product
uv run pytest -m connect
uv run pytest -m workbench
uv run pytest -m package_manager

# Combine markers
uv run pytest -m "performance and connect"

Test categories

Category	Marker	Description
Prerequisites	`prerequisites`	Server reachability, auth, admin onboarding
Package Manager	`package_manager`	CRAN/PyPI mirrors, repos, private packages
Connect	`connect`	Login, deploy, data sources, packages, email
Workbench	`workbench`	Login, IDE launch, sessions, packages
Cross-product	`cross_product`	SSL, monitoring, system resources
Performance	`performance`	Load times, concurrency, resource usage
Security	`security`	HTTPS, auth policy, secrets storage

Version gating

Tests can target specific product versions. Tests are skipped automatically if the deployment is older than required:

@pytest.mark.min_version(product="connect", version="2024.05.0")
def test_new_api_feature():
    ...

Generating reports

After running tests, generate and optionally publish a Quarto report:

# Generate results JSON
uv run pytest --vip-report=report/results.json

# Render the report
cd report && quarto render

# Publish to Connect (optional)
quarto publish connect --server https://connect.example.com

Extending VIP

Add site-specific tests without modifying the VIP source tree. Create a directory with .feature and .py files following the same conventions:

# vip.toml
[general]
extension_dirs = ["/opt/vip-custom-tests"]

# Or via CLI
pytest --vip-extensions=/opt/vip-custom-tests

See examples/custom_tests/ in the repository for a working example.