Exporting Data

Export to Parquet or CSV

Export metrics, events, and project metadata to Parquet or CSV for use in other tools:

velocirepo export ./out/

This writes one file per table:

  out/metrics.parquet (257 KB)
  out/events.parquet (2.9 MB)
  out/content.parquet (15 KB)
  out/indicators.parquet (42 KB)
  out/projects.parquet (2 KB)

Use --format csv for CSV output, and --source or --project to filter:

velocirepo export ./out/ --format csv
velocirepo export ./out/ --source github
velocirepo export ./out/ --project quarto

Using the data with other tools

velocirepo provides two ways to access your metrics data from external tools:

  1. DuckDB file — a persistent velocirepo/data/velocirepo.duckdb file with views over the raw JSONL data and a projects table. Open it directly from any tool that supports DuckDB.
  2. Parquet export — use velocirepo export ./out/ to write Parquet (or CSV) files that any data tool can read.

The DuckDB file is rebuilt automatically after every fetch and project mutation command. You can also rebuild it manually:

velocirepo build-db

Note: The DuckDB file uses relative paths, so external tools must open it from the data directory (or set DuckDB’s working directory to it).

Tool compatibility

Tool / Package DuckDB file Parquet
DuckDB CLI Yes Yes
Python duckdb Yes Yes
Polars (Python/Rust) No Yes
pandas No Yes
Marimo Yes (via duckdb) Yes
R duckdb / DBI Yes Yes
R arrow No Yes
Observable / Evidence Yes Yes
Excel / Google Sheets No Yes (via CSV)

Examples

DuckDB CLI:

cd velocirepo/data
duckdb velocirepo.duckdb "SELECT project, SUM(value) AS stars FROM metrics WHERE metric = 'daily_stars' GROUP BY project ORDER BY stars DESC"

Python (duckdb):

import duckdb
import os

os.chdir("velocirepo/data")
con = duckdb.connect("velocirepo.duckdb", read_only=True)
df = con.sql("SELECT * FROM metrics WHERE source = 'pypi' ORDER BY date DESC LIMIT 10").df()

Polars (from Parquet export):

import polars as pl

metrics = pl.read_parquet("out/metrics.parquet")
stars = metrics.filter(pl.col("metric") == "daily_stars").group_by("project").agg(pl.col("value").sum())

Next steps