Detailed Results

All Test Results

Connect

9 tests — 7 passed, 2 skipped

PASSUser can log in via the web UIConnect
src/vip_tests/connect/test_auth.py::test_connect_login_ui1.63s
Test procedure
  • Given Connect is accessible at the configured URL
  • When a user navigates to the Connect login page
  • And enters valid credentials
  • Then the user is successfully authenticated
  • And the Connect dashboard is displayed
PASSAPI key authentication worksConnect
src/vip_tests/connect/test_auth.py::test_connect_login_api0.40s
Test procedure
  • Given Connect is accessible at the configured URL
  • And a valid API key is configured
  • When I request the current user via the API
  • Then the API returns user information
PASSAdmin user exists and has admin privilegesConnect
src/vip_tests/connect/test_users.py::test_admin_user0.40s
Test procedure
  • Given Connect is accessible at the configured URL
  • When I retrieve the current user profile
  • Then the user has admin privileges
PASSUsers can be listedConnect
src/vip_tests/connect/test_users.py::test_list_users0.40s
Test procedure
  • Given Connect is accessible at the configured URL
  • When I list all users
  • Then the user list is not empty
  • And the test user exists in the user list
PASSGroups can be listedConnect
src/vip_tests/connect/test_users.py::test_list_groups0.40s
Test procedure
  • Given Connect is accessible at the configured URL
  • When I list all groups
  • Then the response is successful
SKIPExpected R versions are available on ConnectConnect
No expected R versions specified in vip.toml [runtimes]
src/vip_tests/connect/test_runtime_versions.py::test_r_versions0.20s
Test procedure
  • Given Connect is accessible at the configured URL
  • And expected R versions are specified in vip.toml
  • When I query Connect for available R versions
  • Then all expected R versions are present
SKIPExpected Python versions are available on ConnectConnect
No expected Python versions specified in vip.toml [runtimes]
src/vip_tests/connect/test_runtime_versions.py::test_python_versions0.20s
Test procedure
  • Given Connect is accessible at the configured URL
  • And expected Python versions are specified in vip.toml
  • When I query Connect for available Python versions
  • Then all expected Python versions are present
PASSExpected Quarto versions are availableConnect
src/vip_tests/connect/test_runtime_versions.py::test_quarto_versions0.40s
Test procedure
  • Given Connect is accessible at the configured URL
  • When I query Connect for available Quarto versions
  • Then at least one Quarto version is available
PASSConnect system checks can be run and the report downloadedConnect
src/vip_tests/connect/test_system_checks.py::test_connect_system_checks26.40s
Test procedure
  • Given Connect is accessible at the configured URL
  • And a valid API key is configured
  • When I trigger a new system check run via the Connect API
  • Then the system check report is returned
  • And I can download the system check report artifact

Package Manager

5 tests — 2 passed, 3 skipped

SKIPPyPI mirror is accessiblePackage Manager
PyPI package 'requests' not available in any of ['pypi-all'] — repo may not be synced yet
src/vip_tests/package_manager/test_repos.py::test_pypi_mirror0.02s
Test procedure
  • Given Package Manager is running
  • When I query the PyPI repository for the "requests" package
  • Then the package is found in the repository
SKIPBioconductor mirror is accessiblePackage Manager
No Bioconductor repository configured in Package Manager
src/vip_tests/package_manager/test_repos.py::test_bioconductor_mirror0.01s
Test procedure
  • Given Package Manager is running
  • When I query the Bioconductor repository for the "BiocGenerics" package
  • Then the package is found in the repository
SKIPOpenVSX mirror is accessiblePackage Manager
No OpenVSX repository configured in Package Manager
src/vip_tests/package_manager/test_repos.py::test_openvsx_mirror0.01s
Test procedure
  • Given Package Manager is running
  • When I query the OpenVSX repository for the "golang.Go" extension
  • Then the package is found in the repository
PASSAt least one repository is configuredPackage Manager
src/vip_tests/package_manager/test_repos.py::test_repo_exists0.01s
Test procedure
  • Given Package Manager is running
  • When I list all repositories
  • Then at least one repository exists
PASSCRAN mirror is accessiblePackage Manager
src/vip_tests/package_manager/test_repos.py::test_cran_mirror1.29s
Test procedure
  • Given Package Manager is running
  • When I query the CRAN repository for the "Matrix" package
  • Then the package is found in the repository

Prerequisites

6 tests — 5 passed, 1 failed

PASSWorkbench server is reachablePrerequisites
src/vip_tests/prerequisites/test_components.py::test_product_server_is_reachable[Workbench]0.12s
Test procedure
  • Given <product> is configured in vip.toml
  • When I request the <product> health endpoint
  • Then the server responds with a successful status code
PASSPackage Manager server is reachablePrerequisites
src/vip_tests/prerequisites/test_components.py::test_product_server_is_reachable[Package Manager]0.02s
Test procedure
  • Given <product> is configured in vip.toml
  • When I request the <product> health endpoint
  • Then the server responds with a successful status code
FAILThis check intentionally fails to demonstrate failure renderingPrerequisites
As someone evaluating a VIP report
src/vip_tests/prerequisites/test_expected_failure.py::test_intentional_failure0.00s
Test procedure
  • Given a check that is written to fail on purpose
  • When the check runs as part of this example report
  • Then it fails by design, not because anything is actually broken
test_intentional_failure: This failure is intentional: it exists only to show how a failed check renders in this example report. No product configuration will resolve it. assert 'fail (by design)' == 'pass' - pass + fail (by design)
Full error traceback
fixturefunc = <function check_fails_by_design at 0x7f1e232487d0>
request = <FixtureRequest for <Function test_intentional_failure>>
kwargs = {'demo_outcome': {'expected': 'pass', 'actual': 'fail (by design)'}}

    def call_fixture_func(
        fixturefunc: _FixtureFunc[FixtureValue], request: FixtureRequest, kwargs
    ) -> FixtureValue:
        if inspect.isgeneratorfunction(fixturefunc):
            fixturefunc = cast(Callable[..., Generator[FixtureValue]], fixturefunc)
            generator = fixturefunc(**kwargs)
            try:
                fixture_result = next(generator)
            except StopIteration:
                raise ValueError(f"{request.fixturename} did not yield a value") from None
            finalizer = functools.partial(_teardown_yield_fixture, fixturefunc, generator)
            request.addfinalizer(finalizer)
        else:
            fixturefunc = cast(Callable[..., FixtureValue], fixturefunc)
>           fixture_result = fixturefunc(**kwargs)
                             ^^^^^^^^^^^^^^^^^^^^^

.venv/lib/python3.14/site-packages/_pytest/fixtures.py:1005: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

demo_outcome = {'expected': 'pass', 'actual': 'fail (by design)'}

    @then("it fails by design, not because anything is actually broken")
    def check_fails_by_design(demo_outcome):
>       assert demo_outcome["actual"] == demo_outcome["expected"], (
            "This failure is intentional: it exists only to show how a failed check "
            "renders in this example report. No product configuration will resolve it."
        )
E       AssertionError: This failure is intentional: it exists only to show how a failed check renders in this example report. No product configuration will resolve it.
E       assert 'fail (by design)' == 'pass'
E         
E         - pass
E         + fail (by design)

src/vip_tests/prerequisites/test_expected_failure.py:49: AssertionError
Likely causes:
  • This check is designed to always fail, so the example report has a failure card to show
  • No product or configuration problem is involved
Suggested next steps:
  1. No action needed. This failure is intentional and does not indicate a real problem
  2. It only runs when VIP_ENABLE_EXPECTED_FAILURE_DEMO=1 is set

Documentation: https://github.com/posit-dev/vip/issues/73

PASSConnect version matches configurationPrerequisites
src/vip_tests/prerequisites/test_versions.py::test_connect_version0.20s
Test procedure
  • Given Connect is configured in vip.toml with a version expectation
  • When I fetch the Connect server version
  • Then the Connect version matches the configured value
PASSPackage Manager version matches configurationPrerequisites
src/vip_tests/prerequisites/test_versions.py::test_package_manager_version0.02s
Test procedure
  • Given Package Manager is configured in vip.toml with a version expectation
  • When I fetch the Package Manager server version
  • Then the Package Manager version matches the configured value
PASSConnect server is reachablePrerequisites
src/vip_tests/prerequisites/test_components.py::test_product_server_is_reachable[Connect]0.01s
Test procedure
  • Given <product> is configured in vip.toml
  • When I request the <product> health endpoint
  • Then the server responds with a successful status code

Security

3 tests — 3 passed

PASSUnauthenticated API request returns 401Security
src/vip_tests/security/test_error_handling.py::test_unauthenticated_api_request_returns_4010.01s
Test procedure
  • Given Connect is configured in vip.toml
  • When I make an unauthenticated API request to Connect
  • Then the response status is 401
PASSInvalid API key returns 401Security
src/vip_tests/security/test_error_handling.py::test_invalid_api_key_returns_4014.30s
Test procedure
  • Given Connect is configured in vip.toml
  • When I make an API request to Connect with an invalid key
  • Then the response status is 401
PASSNon-existent endpoint returns 404Security
src/vip_tests/security/test_error_handling.py::test_nonexistent_endpoint_returns_4040.01s
Test procedure
  • Given Connect is configured in vip.toml
  • When I request a non-existent endpoint on Connect
  • Then the response status is 404

Workbench

2 tests — 2 passed

PASSUser can sign out of WorkbenchWorkbench
src/vip_tests/workbench/test_auth.py::test_workbench_signout[chromium]0.78s
Test procedure
  • Given Workbench is accessible and I am logged in
  • When I sign out of Workbench
  • Then I am redirected to the Workbench login page
PASSUser can log in to Workbench via the web UIWorkbench
src/vip_tests/workbench/test_auth.py::test_workbench_login[chromium]5.21s
Test procedure
  • Given Workbench is accessible at the configured URL
  • When a user navigates to the Workbench login page and enters valid credentials
  • Then the Workbench homepage is displayed
  • And the current user element is visible and non-empty in the header