Getting Started

Quick start

Install VIP, then point it at your server:

uv venv
source .venv/bin/activate
uv pip install posit-vip
playwright install chromium

Using RStudio or Positron?

You can set up VIP as an RStudio project instead. Go to File → New Project → Version Control → Git and enter https://github.com/posit-dev/vip.git as the Repository URL. Then install dependencies from the Terminal tab:

uv sync
uv run playwright install chromium

See the Shiny app page for the full graphical workflow.

Run your first test against a Connect server:

vip verify --connect-url https://connect.example.com

A browser window opens for you to log in. After authentication completes, VIP runs the test suite headlessly and cleans up the session automatically.

Test multiple products at once:

vip verify \
  --connect-url https://connect.example.com \
  --workbench-url https://workbench.example.com \
  --pm-url https://packagemanager.example.com

Authentication

Interactive auth (default)

VIP opens a Chromium window so you can log in through any identity provider (Okta, SAML, OIDC, password). After login, VIP mints a temporary API key, saves the session, and runs all tests headlessly. No credentials need to be configured.

# Interactive auth is the default — just run:
vip verify --connect-url https://connect.example.com

Credential-based auth

If you have API keys or test user credentials, disable interactive auth and set environment variables instead:

export VIP_CONNECT_API_KEY="your-api-key"
export VIP_TEST_USERNAME="test-user"
export VIP_TEST_PASSWORD="test-password"
vip verify --connect-url https://connect.example.com --no-interactive-auth

Running tests

Prefer a graphical interface? VIP includes a Shiny app that lets you select categories, run tests, and view the report from your browser or the Workbench Viewer pane.

Target specific products or test categories:

# Run all tests
vip verify --connect-url https://connect.example.com

# Run tests for a specific category
vip verify --connect-url https://connect.example.com --categories connect
vip verify --workbench-url https://workbench.example.com --categories workbench
vip verify --pm-url https://pm.example.com --categories package_manager

# Combine markers
vip verify --connect-url https://connect.example.com --categories "performance and connect"

# Pass extra flags to pytest
vip verify --connect-url https://connect.example.com -- -x --tb=long -k 'login'

Test categories

Version gating

Tests can target specific product versions. Tests are skipped automatically if the deployment is older than required:

@pytest.mark.min_version(product="connect", version="2024.05.0")
def test_new_api_feature():
    ...

Configuration file

For repeated testing or advanced configuration, use a vip.toml file instead of URL flags:

cp vip.toml.example vip.toml
# Edit vip.toml with your deployment details

Then run tests against it:

vip verify --config vip.toml --no-interactive-auth

If no --config is specified and no URL flags are given, VIP looks for vip.toml in the current directory or the path in the VIP_CONFIG environment variable.

Each product section (Connect, Workbench, Package Manager) can be enabled or disabled individually. See vip.toml.example for the full template.

Generating reports

After running tests, generate and optionally publish a Quarto report:

# Generate results JSON
vip verify --connect-url https://connect.example.com --report report/results.json

# Render the report
cd report && quarto render

# Publish to Connect (optional)
quarto publish connect --server https://connect.example.com

Extending VIP

Add site-specific tests without modifying the VIP source tree. Create a directory with .feature and .py files following the same conventions:

vip verify --connect-url https://connect.example.com \
  --extensions /opt/vip-custom-tests

Or configure extension directories in vip.toml:

# vip.toml
[general]
extension_dirs = ["/opt/vip-custom-tests"]

See examples/custom_tests/ in the repository for a working example.