IN DEVELOPMENT · PUBLIC ENGINEERING LABRodrigo Manso · supported by DeepCode

F1 Data Platform: from telemetry to AI experiments.

Bringing industrial data engineering to a motorsport laboratory: historical data, game telemetry, reliable replay and AI-assisted lap experiments. Eight connected components, built and explained in public.

GitHub ↗ Milestones and evidence

Status · 13 October 2026. ETL implementation, a Rust historian v1 and isolated Python prototypes are in progress. The integrated live pipeline, AI lap optimisation and motion rig have not been demonstrated. The architecture below is the target design; repository code and evidence determine completion.

target architecture

Two data paths. One evidence trail.

Historical data

Jolpica → Python ETL → DuckDB / Parquet → statistics tools + RAG

FastF1 → batch import → box-box → comparative dashboards

Live and replayed telemetry

F1 game / recorded capture → UDP capture + fan-out → edge connector → box-box → dashboard

The capture path must preserve session identity, source timestamps and data quality. Durable recovery is a planned requirement; the current buffer is in memory.

AI works between laps. Analyse a recorded lap → propose a bounded script edit → validate it → execute through a deterministic controller → record the result. Model requests stay outside the real-time control loop. Distance indexing helps alignment; it does not prove stable driving.

Game and real-world laps can be aligned for exploration, but they have different physics, conditions and sampling. Motion cues derived from public data are an optional research extension, not a reproduction of actual driver forces.

implementation map

What exists, and what comes next

A monorepo with eight cases. Prototype code is evidence of progress, not a claim of production readiness.

f1-data-platform

Python · DuckDB · Parquet

Implementation in progress: layered historical ETL. The component README tracks execution evidence and remaining backfill.

box-box

Rust · Arrow/Parquet

v1 implemented and exercised locally: HTTP ingest with validation, per-tag swinging-door compression, WAL + Parquet segments, restart recovery and a query API with downsampling. Pending: FastF1 batch importer, per-tag retention and runs against real game packets.

f1-sim-connector

Python · UDP

Header parsing, tag mapping, exception filtering and an in-memory buffer prototype. Packet-body decoding, UDP capture and historian integration remain to be built.

f1-telemetry-lab

Python · Streamlit

Dashboard scaffold. Planned comparisons by lap distance, with source, quality and replay state visible.

f1-history-agent

Python · FastAPI · RAG + SQL

Health endpoint scaffold. Planned read-only statistics tools, cited narrative retrieval and a versioned evaluation set.

f1-sim-driver

Python · structured lap scripts

Script validation, distance-based interpolation, position-hold and edit prototypes. No connected game-control loop yet; repeatability and input adapters need validation.

f1-strategy-ml

Python · scikit-learn

Later phase: degradation models and simulator-to-real-data experiments. Generalisation is a research question, not an established result.

f1-mobile

Flutter · Dart

Later phase: a client for stable platform APIs. Motion-rig research is a separate, optional extension.

acceptance gates

One demonstrable slice at a time

  1. Capture and explain: one recorded session, versioned packet fixtures, session identity and timestamps.
  2. Store and replay: a visible capture → connector → historian → dashboard path, with restart, outage and loss tests.
  3. Measure fidelity: raw versus filtered/compressed signals, event timing, latency, memory and recovery. Publish the test environment.
  4. Establish a baseline: repeatable non-LLM control, validated adapters, lap validity and failure handling.
  5. Evaluate AI edits: bounded changes between runs, equal budgets, repeated trials and all failed laps reported.

Validation and benchmark protocol ↗

build journal · planned series

Motorsport Engineering Lab

A series by Rodrigo Manso, supported by DeepCode. Videos in English for YouTube and Instagram, following actual implementation milestones. Portuguese subtitles may be added when useful.

First chapters

Why this lab • Reading the first packets • Sessions, clocks and data quality • Recording and replaying a lap.

Then, the experiments

Storage and recovery • Compression versus fidelity • A repeatable baseline • AI edits with measured outcomes and visible failures.

Follow the series on YouTube and Instagram. Episodes are in preparation. Explore the code and milestones on GitHub.

Sources and boundaries

Repository code and data have different licenses. Jolpica data is subject to CC BY-NC-SA 4.0; FastF1’s MIT software license does not establish blanket rights to underlying telemetry. Each dataset and any published footage needs its own attribution and permitted-use review.

Jolpica terms · Project foundations