Detailed Results

All Test Results

Connect

9 tests — 7 passed, 2 skipped

PASSUser can log in via the web UIConnect

src/vip_tests/connect/test_auth.py::test_connect_login_ui1.75s

Test procedure

Given Connect is accessible at the configured URL
When a user navigates to the Connect login page
And enters valid credentials
Then the user is successfully authenticated
And the Connect dashboard is displayed

PASSAPI key authentication worksConnect

src/vip_tests/connect/test_auth.py::test_connect_login_api0.39s

Test procedure

Given Connect is accessible at the configured URL
And a valid API key is configured
When I request the current user via the API
Then the API returns user information

PASSAdmin user exists and has admin privilegesConnect

src/vip_tests/connect/test_users.py::test_admin_user0.39s

Test procedure

Given Connect is accessible at the configured URL
When I retrieve the current user profile
Then the user has admin privileges

PASSUsers can be listedConnect

src/vip_tests/connect/test_users.py::test_list_users0.39s

Test procedure

Given Connect is accessible at the configured URL
When I list all users
Then the user list is not empty
And the test user exists in the user list

PASSGroups can be listedConnect

src/vip_tests/connect/test_users.py::test_list_groups0.39s

Test procedure

Given Connect is accessible at the configured URL
When I list all groups
Then the response is successful

SKIPExpected R versions are available on ConnectConnect

No expected R versions specified in vip.toml [runtimes]

src/vip_tests/connect/test_runtime_versions.py::test_r_versions0.20s

Test procedure

Given Connect is accessible at the configured URL
And expected R versions are specified in vip.toml
When I query Connect for available R versions
Then all expected R versions are present

SKIPExpected Python versions are available on ConnectConnect

No expected Python versions specified in vip.toml [runtimes]

src/vip_tests/connect/test_runtime_versions.py::test_python_versions0.20s

Test procedure

Given Connect is accessible at the configured URL
And expected Python versions are specified in vip.toml
When I query Connect for available Python versions
Then all expected Python versions are present

PASSExpected Quarto versions are availableConnect

src/vip_tests/connect/test_runtime_versions.py::test_quarto_versions0.39s

Test procedure

Given Connect is accessible at the configured URL
When I query Connect for available Quarto versions
Then at least one Quarto version is available

PASSConnect system checks can be run and the report downloadedConnect

src/vip_tests/connect/test_system_checks.py::test_connect_system_checks29.53s

Test procedure

Given Connect is accessible at the configured URL
And a valid API key is configured
When I trigger a new system check run via the Connect API
Then the system check report is returned
And I can download the system check report artifact

Package Manager

5 tests — 2 passed, 3 skipped

SKIPPyPI mirror is accessiblePackage Manager

PyPI package 'requests' not available in any of ['pypi-all'] — repo may not be synced yet

src/vip_tests/package_manager/test_repos.py::test_pypi_mirror0.02s

Test procedure

Given Package Manager is running
When I query the PyPI repository for the "requests" package
Then the package is found in the repository

SKIPBioconductor mirror is accessiblePackage Manager

No Bioconductor repository configured in Package Manager

src/vip_tests/package_manager/test_repos.py::test_bioconductor_mirror0.01s

Test procedure

Given Package Manager is running
When I query the Bioconductor repository for the "BiocGenerics" package
Then the package is found in the repository

SKIPOpenVSX mirror is accessiblePackage Manager

No OpenVSX repository configured in Package Manager

src/vip_tests/package_manager/test_repos.py::test_openvsx_mirror0.01s

Test procedure

Given Package Manager is running
When I query the OpenVSX repository for the "golang.Go" extension
Then the package is found in the repository

PASSAt least one repository is configuredPackage Manager

src/vip_tests/package_manager/test_repos.py::test_repo_exists0.01s

Test procedure

Given Package Manager is running
When I list all repositories
Then at least one repository exists

PASSCRAN mirror is accessiblePackage Manager

src/vip_tests/package_manager/test_repos.py::test_cran_mirror1.04s

Test procedure

Given Package Manager is running
When I query the CRAN repository for the "Matrix" package
Then the package is found in the repository

Prerequisites

6 tests — 5 passed, 1 failed

PASSWorkbench server is reachablePrerequisites

src/vip_tests/prerequisites/test_components.py::test_product_server_is_reachable[Workbench]0.11s

Test procedure

Given <product> is configured in vip.toml
When I request the <product> health endpoint
Then the server responds with a successful status code

PASSPackage Manager server is reachablePrerequisites

src/vip_tests/prerequisites/test_components.py::test_product_server_is_reachable[Package Manager]0.02s

Test procedure

Given <product> is configured in vip.toml
When I request the <product> health endpoint
Then the server responds with a successful status code

FAILThis check intentionally fails to demonstrate failure renderingPrerequisites

As someone evaluating a VIP report

src/vip_tests/prerequisites/test_expected_failure.py::test_intentional_failure0.00s

Test procedure

Given a check that is written to fail on purpose
When the check runs as part of this example report
Then it fails by design, not because anything is actually broken

test_intentional_failure: This failure is intentional: it exists only to show how a failed check renders in this example report. No product configuration will resolve it. assert 'fail (by design)' == 'pass' - pass + fail (by design)

Full error traceback

fixturefunc = <function check_fails_by_design at 0x7fa75670c7d0>
request = <FixtureRequest for <Function test_intentional_failure>>
kwargs = {'demo_outcome': {'expected': 'pass', 'actual': 'fail (by design)'}}

    def call_fixture_func(
        fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
    ) -> FixtureValue:
        if inspect.isgeneratorfunction(fixturefunc):
            fixturefunc = cast(Callable[..., Generator[FixtureValue]], fixturefunc)
            generator = fixturefunc(**kwargs)
            try:
                fixture_result = next(generator)
            except StopIteration:
                raise ValueError(f"{request.fixturename} did not yield a value") from None
            finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
            request.addfinalizer(finalizer)
        else:
            fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
>           fixture_result = fixturefunc(**kwargs)
                             ^^^^^^^^^^^^^^^^^^^^^

.venv/lib/python3.14/site-packages/_pytest/fixtures.py:1005: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

demo_outcome = {'expected': 'pass', 'actual': 'fail (by design)'}

    @then("it fails by design, not because anything is actually broken")
    def check_fails_by_design(demo_outcome):
>       assert demo_outcome["actual"] == demo_outcome["expected"], (
            "This failure is intentional: it exists only to show how a failed check "
            "renders in this example report. No product configuration will resolve it."
        )
E       AssertionError: This failure is intentional: it exists only to show how a failed check renders in this example report. No product configuration will resolve it.
E       assert 'fail (by design)' == 'pass'
E         
E         - pass
E         + fail (by design)

src/vip_tests/prerequisites/test_expected_failure.py:49: AssertionError

Likely causes:

This check is designed to always fail, so the example report has a failure card to show
No product or configuration problem is involved

Suggested next steps:

No action needed. This failure is intentional and does not indicate a real problem
It only runs when VIP_ENABLE_EXPECTED_FAILURE_DEMO=1 is set

Documentation: https://github.com/posit-dev/vip/issues/73

PASSConnect version matches configurationPrerequisites

src/vip_tests/prerequisites/test_versions.py::test_connect_version0.20s

Test procedure

Given Connect is configured in vip.toml with a version expectation
When I fetch the Connect server version
Then the Connect version matches the configured value

PASSPackage Manager version matches configurationPrerequisites

src/vip_tests/prerequisites/test_versions.py::test_package_manager_version0.02s

Test procedure

Given Package Manager is configured in vip.toml with a version expectation
When I fetch the Package Manager server version
Then the Package Manager version matches the configured value

PASSConnect server is reachablePrerequisites

src/vip_tests/prerequisites/test_components.py::test_product_server_is_reachable[Connect]0.01s

Test procedure

Given <product> is configured in vip.toml
When I request the <product> health endpoint
Then the server responds with a successful status code

Security

3 tests — 3 passed

PASSUnauthenticated API request returns 401Security

src/vip_tests/security/test_error_handling.py::test_unauthenticated_api_request_returns_4010.01s

Test procedure

Given Connect is configured in vip.toml
When I make an unauthenticated API request to Connect
Then the response status is 401

PASSInvalid API key returns 401Security

src/vip_tests/security/test_error_handling.py::test_invalid_api_key_returns_4014.00s

Test procedure

Given Connect is configured in vip.toml
When I make an API request to Connect with an invalid key
Then the response status is 401

PASSNon-existent endpoint returns 404Security

src/vip_tests/security/test_error_handling.py::test_nonexistent_endpoint_returns_4040.01s

Test procedure

Given Connect is configured in vip.toml
When I request a non-existent endpoint on Connect
Then the response status is 404

Workbench

2 tests — 2 passed

PASSUser can sign out of WorkbenchWorkbench

src/vip_tests/workbench/test_auth.py::test_workbench_signout[chromium]0.75s

Test procedure

Given Workbench is accessible and I am logged in
When I sign out of Workbench
Then I am redirected to the Workbench login page

PASSUser can log in to Workbench via the web UIWorkbench

src/vip_tests/workbench/test_auth.py::test_workbench_login[chromium]5.17s

Test procedure

Given Workbench is accessible at the configured URL
When a user navigates to the Workbench login page and enters valid credentials
Then the Workbench homepage is displayed
And the current user element is visible and non-empty in the header