| Product | URL | Version | Results |
|---|---|---|---|
| Connect | http://localhost:3939 | 2026.08.1 | 7 passed, 2 skipped (9 tests) |
| Workbench | http://localhost:8787 | 2026.08.2+200.pro1 | 1 passed, 1 failed (2 tests) |
| Package Manager | http://localhost:4242 | 2026.08.0 | 2 passed, 3 skipped (5 tests) |
VIP Report
Verified Installation of Posit
Deployment: CI Connect 2026.08.1 + WB 2026.08.2+200.pro1 + PM 2026.08.0 Generated: 2026-09-01 00:34:18 UTC
Products Under Test
Summary
| Total | 25 |
|---|---|
| Passed | 18 |
| Failed | 2 |
| Skipped | 5 |
| Status | FAIL |
Provenance
| VIP version | 2026.8.3 |
|---|---|
| Run duration | 41.2s |
| Python | 3.14.7 |
| Platform | Linux-6.17.0-1022-azure-x86_64-with-glibc2.39 |
| Mode | full |
| Exit status | 1 (tests were collected and run, but some failed) |
| Performed by | bdeitte (GitHub actor) |
| Run host | runnervmgx7h7 |
| Commit | 18555cb7140ccc2d37eb487a32ede1b5f7675d06 |
| Branch | 627/merge |
| CI run | https://github.com/posit-dev/vip/actions/runs/33454784077 |
Failures & Skips
Every check that did not pass, in full. Passing checks are counted above — see Detailed Results for the complete listing.
Failed (2)
FAILUser can log in to Workbench via the web UIWorkbench
Test procedure
- Given Workbench is accessible at the configured URL
- When a user navigates to the Workbench login page and enters valid credentials
- Then the Workbench homepage is displayed
- And the current user element is visible and non-empty in the header
test_workbench_login[chromium]: Login failed: Error: Temporary server error, please try again
Full error traceback
fixturefunc = <function navigate_and_login at 0x7f62293c7e20>
request = <FixtureRequest for <Function test_workbench_login[chromium]>>
kwargs = {'page': <Page url='http://localhost:8787/auth-sign-in?appUri=%2F&error=2'>, 'workbench_url': 'http://localhost:8787', 'test_username': 'testuser', 'test_password': 'swLU1F8H2xeF6YgA'}
def call_fixture_func(
fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
) -> FixtureValue:
if inspect.isgeneratorfunction(fixturefunc):
fixturefunc = cast(Callable[..., Generator[FixtureValue]], fixturefunc)
generator = fixturefunc(**kwargs)
try:
fixture_result = next(generator)
except StopIteration:
raise ValueError(f"{request.fixturename} did not yield a value") from None
finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
request.addfinalizer(finalizer)
else:
fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
> fixture_result = fixturefunc(**kwargs)
^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.14/site-packages/_pytest/fixtures.py:1005:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
src/vip_tests/workbench/test_auth.py:105: in navigate_and_login
workbench_login(page, workbench_url, test_username, test_password)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
page = <Page url='http://localhost:8787/auth-sign-in?appUri=%2F&error=2'>
workbench_url = 'http://localhost:8787', username = 'testuser'
password = 'swLU1F8H2xeF6YgA', auth_provider = 'password'
interactive_auth = False, auth_mode = 'none', workbench_auth_error = None
max_retries = 3, retry_delay = 2.0
def workbench_login(
page: Page,
workbench_url: str,
username: str,
password: str,
auth_provider: str = "password",
interactive_auth: bool = False,
*,
auth_mode: str = "none",
workbench_auth_error: str | None = None,
max_retries: int = 3,
retry_delay: float = 2.0,
) -> None:
"""Navigate to Workbench homepage, logging in only if required.
This function:
- Navigates directly to Workbench's URL
- Handles OIDC/SSO via pre-loaded storage state (--interactive-auth / --headless-auth)
- Only fills login form for password auth
- Retries on transient server errors (e.g., too many logins)
Args:
page: Playwright page object
workbench_url: Base URL for Workbench (e.g., http://localhost:8787)
username: Login username
password: Login password
auth_provider: Auth type (e.g., "password", "oidc", "saml")
interactive_auth: True when an auth session is pre-loaded (either
--interactive-auth or --headless-auth)
auth_mode: Active auth mode ("interactive", "headless", or "none"),
used to name the responsible CLI flag in skip messages
workbench_auth_error: Reason the pre-test auth flow could not
establish a Workbench session, if known. Quoted in the skip
message so users see the real cause instead of a guess.
max_retries: Max login attempts on transient errors (default 3)
retry_delay: Seconds to wait between retries (default 2.0)
Raises:
pytest.skip: For non-password auth without a pre-loaded auth session,
or when the session's storage state doesn't cover Workbench
AssertionError: When password login fails after retries
"""
homepage_logo = page.locator(Homepage.POSIT_LOGO)
# For non-password auth without a pre-loaded auth session, skip immediately
if auth_provider != "password" and not interactive_auth:
pytest.skip(
f"Login form not available for auth provider {auth_provider!r}. "
"Pass --interactive-auth or --headless-auth to pre-load browser storage state."
)
page.goto(workbench_url)
page.wait_for_load_state("load")
# Fast path: already logged in (common with interactive_auth)?
if homepage_logo.is_visible():
return
# Check if we landed on a login/IdP page
if _on_login_page(page.url):
# The sign-in page renders client-side after ``load``; wait once for
# either the password form's username field or an OIDC "Sign in with ..."
# button to appear before deciding which flow applies. A short fixed wait
# on only the SSO button races the render and can misread a slow OIDC
# sign-in page (e.g. an ``?error=2`` bounce) as a password deployment --
# which then fails the retry loop with "Login failed after 3 attempts"
# instead of skipping. Waiting for either control settles that race
# without penalising real password deployments (the username field
# appears promptly there).
try:
page.locator(f"{LoginPage.USERNAME}, button:has-text('Sign in')").first.wait_for(
state="visible", timeout=TIMEOUT_PAGE_LOAD
)
except Exception:
pass
# An OIDC sign-in page shows a "Sign in with ..." button and no username
# field. The "sign in" role-name also matches a password form's submit
# button, so the *absence* of the username field is what distinguishes a
# true SSO-only page from a password form.
sso_button = page.get_by_role("button", name=re.compile(r"sign in", re.IGNORECASE)).first
sso_button_present = sso_button.is_visible()
# A deployment can also skip its own sign-in page entirely and redirect
# straight to the IdP, whose markup matches neither probe above (see
# _external_idp_host). Landing off the Workbench origin is conclusive on
# its own: no Workbench password form is reachable from there.
idp_host = _external_idp_host(page.url, workbench_url)
sso_only = idp_host is not None or (
sso_button_present and not page.locator(LoginPage.USERNAME).is_visible()
)
if interactive_auth and sso_only:
# Storage state was pre-loaded by --interactive-auth / --headless-auth.
# Workbench's SSO sign-in page does not auto-redirect to the IdP; it renders a
# "Sign in with OpenID" button. Clicking it triggers a silent SSO round-trip
# using the saved IdP cookies. The round-trip is serialized across xdist
# workers (see _silent_sso_signin / oidc_login_lock) to avoid storming the
# shared IdP session (#484/#467). When the deployment redirected us
# off-origin instead there is no such button to click, so go straight
# to the skip -- clicking a locator that resolves to nothing would
# raise a Playwright error in place of an actionable skip.
if sso_button_present and _silent_sso_signin(sso_button, homepage_logo, workbench_url):
return # Silent SSO succeeded
# No usable IdP session (expired, or storage state was stripped for the
# password-login test) — skip gracefully. Recompute the IdP host: the
# click above can navigate off-origin before timing out, so where we
# ended up is only knowable now, not before the attempt.
_skip_workbench_session_unproven(
auth_mode=auth_mode,
workbench_auth_error=workbench_auth_error,
landed_url=page.url,
idp_host=_external_idp_host(page.url, workbench_url),
)
if auth_provider != "password":
_skip_workbench_session_unproven(
auth_mode=auth_mode,
workbench_auth_error=workbench_auth_error,
landed_url=page.url,
idp_host=_external_idp_host(page.url, workbench_url),
)
# Even when auth_provider is reported as "password", the deployment may
# actually present an SSO/OIDC sign-in page (a "Sign in with ..." button
# and no username field) — e.g. auth_provider defaulted to "password" on
# a config-less run against an OIDC deployment. The password login form
# is unavailable there, so skip rather than fail the retry loop.
if sso_only:
where = (
f"redirects sign-in to the identity provider at {idp_host}"
if idp_host
else "presents an SSO/OIDC sign-in page instead (no username/password fields)"
)
pytest.skip(
"This scenario requires a Workbench password-login form, but the "
f"deployment {where}. Password login cannot be exercised on an SSO "
"deployment, so this scenario is skipped."
)
# Password auth - proceed with form login below
else:
# Not on homepage, not on login page - unexpected state
# Give it one more check in case page is still loading
try:
homepage_logo.wait_for(state="visible", timeout=TIMEOUT_QUICK)
return
except Exception:
pass
# Password authentication with retry logic
login_form = page.locator(LoginPage.USERNAME)
error_panel = page.locator(LoginPage.ERROR_PANEL)
for attempt in range(max_retries):
if attempt > 0:
time.sleep(retry_delay)
page.goto(workbench_url)
# Fast path check on retry
if homepage_logo.is_visible():
return
# Wait for login form to be ready
try:
login_form.wait_for(state="visible", timeout=TIMEOUT_QUICK)
except Exception:
continue
# Fill and submit
page.fill(LoginPage.USERNAME, username)
page.fill(LoginPage.PASSWORD, password)
stay_signed_in = page.locator(LoginPage.STAY_SIGNED_IN)
if stay_signed_in.is_visible() and not stay_signed_in.is_checked():
stay_signed_in.click()
page.click(LoginPage.BUTTON)
# Wait for either homepage (success) or error panel (failure)
homepage_or_error = homepage_logo.or_(error_panel)
try:
homepage_or_error.wait_for(state="visible", timeout=TIMEOUT_PAGE_LOAD)
except Exception:
if attempt == max_retries - 1:
raise AssertionError(f"Login failed after {max_retries} attempts: no response")
continue
# Check which one appeared
if homepage_logo.is_visible():
return # Success!
# Error appeared - extract message and maybe retry
if attempt == max_retries - 1:
error_text = page.locator(LoginPage.ERROR_TEXT).text_content()
> raise AssertionError(f"Login failed: {error_text or 'Unknown error'}")
E AssertionError: Login failed: Error: Temporary server error, please try again
src/vip_tests/workbench/conftest.py:937: AssertionErrorLikely causes:
- Incorrect test credentials in vip.toml or VIP_TEST_USERNAME/VIP_TEST_PASSWORD
- Auth provider misconfiguration (SAML, OIDC, LDAP, PAM)
- Workbench login page markup changed or is unreachable
- Verify credentials work by logging in manually at the Workbench URL
- Check the auth provider configuration in rserver.conf / auth-pam or auth-saml settings
- If using an external IdP, try `--interactive-auth` or `--headless-auth` instead of a password test
Documentation: https://docs.posit.co/ide/server-pro/authenticating_users/authenticating_users.html
FAILThis check intentionally fails to demonstrate failure renderingPrerequisites
As someone evaluating a VIP report
Test procedure
- Given a check that is written to fail on purpose
- When the check runs as part of this example report
- Then it fails by design, not because anything is actually broken
test_intentional_failure: This failure is intentional: it exists only to show how a failed check renders in this example report. No product configuration will resolve it.
assert 'fail (by design)' == 'pass'
- pass
+ fail (by design)
Full error traceback
fixturefunc = <function check_fails_by_design at 0x7f6229385010>
request = <FixtureRequest for <Function test_intentional_failure>>
kwargs = {'demo_outcome': {'expected': 'pass', 'actual': 'fail (by design)'}}
def call_fixture_func(
fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
) -> FixtureValue:
if inspect.isgeneratorfunction(fixturefunc):
fixturefunc = cast(Callable[..., Generator[FixtureValue]], fixturefunc)
generator = fixturefunc(**kwargs)
try:
fixture_result = next(generator)
except StopIteration:
raise ValueError(f"{request.fixturename} did not yield a value") from None
finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
request.addfinalizer(finalizer)
else:
fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
> fixture_result = fixturefunc(**kwargs)
^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.14/site-packages/_pytest/fixtures.py:1005:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
demo_outcome = {'expected': 'pass', 'actual': 'fail (by design)'}
@then("it fails by design, not because anything is actually broken")
def check_fails_by_design(demo_outcome):
> assert demo_outcome["actual"] == demo_outcome["expected"], (
"This failure is intentional: it exists only to show how a failed check "
"renders in this example report. No product configuration will resolve it."
)
E AssertionError: This failure is intentional: it exists only to show how a failed check renders in this example report. No product configuration will resolve it.
E assert 'fail (by design)' == 'pass'
E
E - pass
E + fail (by design)
src/vip_tests/prerequisites/test_expected_failure.py:49: AssertionErrorLikely causes:
- This check is designed to always fail, so the example report has a failure card to show
- No product or configuration problem is involved
- No action needed. This failure is intentional and does not indicate a real problem
- It only runs when VIP_ENABLE_EXPECTED_FAILURE_DEMO=1 is set
Documentation: https://github.com/posit-dev/vip/issues/73
Skipped (5)
SKIPExpected R versions are available on ConnectConnect
No expected R versions specified in vip.toml [runtimes]
Test procedure
- Given Connect is accessible at the configured URL
- And expected R versions are specified in vip.toml
- When I query Connect for available R versions
- Then all expected R versions are present
SKIPExpected Python versions are available on ConnectConnect
No expected Python versions specified in vip.toml [runtimes]
Test procedure
- Given Connect is accessible at the configured URL
- And expected Python versions are specified in vip.toml
- When I query Connect for available Python versions
- Then all expected Python versions are present
SKIPPyPI mirror is accessiblePackage Manager
PyPI package 'requests' not available in any of ['pypi-all'] — repo may not be synced yet
Test procedure
- Given Package Manager is running
- When I query the PyPI repository for the "requests" package
- Then the package is found in the repository
SKIPBioconductor mirror is accessiblePackage Manager
No Bioconductor repository configured in Package Manager
Test procedure
- Given Package Manager is running
- When I query the Bioconductor repository for the "BiocGenerics" package
- Then the package is found in the repository
SKIPOpenVSX mirror is accessiblePackage Manager
No OpenVSX repository configured in Package Manager
Test procedure
- Given Package Manager is running
- When I query the OpenVSX repository for the "golang.Go" extension
- Then the package is found in the repository