Data Retention

Lightarch uses a two-tier storage model: a fast hot index (D1) for recent data, and unlimited cold storage (R2 + Iceberg) for historical data.

Hot index (D1)

Table	Retention	Notes
recent_spans	48 hours	Spans and logs — instant queries
metrics_samples	7 days	Raw metric data points
usage_events	90 days	Billing events — aggregated to usage_daily after 1 day
alert_events	Indefinite	Alert firing history
usage_daily	Indefinite	Daily aggregated billing totals
dashboards	Indefinite	Your saved dashboards
alert_rules	Indefinite	Your alert rule definitions

D1 is capped per plan to protect the 10GB D1 database limit: Free = 50K rows, Pro = 500K rows, Enterprise = 3M rows. At 1M spans/day (Pro), 48h retention keeps you well under 500K rows.

Cold storage (R2)

All spans flow through a Cloudflare Pipeline to your R2 bucket in Iceberg table format — unlimited retention at R2 storage rates (~$0.015/GB/month). The data is yours, in your bucket.

Querying cold data

When you fetch a trace older than 48h, the query-proxy automatically falls back to R2 SQL:

Method	When	Latency
D1 hot index	Span < 48h old	< 100ms
R2 cold storage	Span > 48h old (auto-fallback)	2–5 seconds

The UI shows "⚠ Fetched from cold storage (R2)" in the waterfall when a cold fetch occurs. The CLI shows source: cold in JSON output.

Cleanup schedule

A nightly cron job (eo-billing-sync, runs at 00:05 UTC) deletes expired rows in small batches of 500 to avoid holding write locks during active ingest. The delete runs while the system is live — ingest is never blocked for more than ~20ms at a stretch.

Exporting your data

Since your data lives in your own R2 bucket in Iceberg format, you can query or export it directly using any Iceberg-compatible tool (Spark, DuckDB, Trino, etc.).