Crack the Sky
Repo layout
Top level
Four Python packages plus tests and ops scripts. Jobs are
python -m ingest.jobs.<name> (or
python -m pricing.term_structure /
python -m signals.spot) from the venv
(pip install -e . so imports do not depend on cwd). No
console-script entry points.
| Path | What it is |
|---|---|
| ingest/ | Capture. REST, delayed websocket, S3 flat files, schemas, cron jobs, entitlements probe. |
| marketdata/ | Typed reads of clean parquet. OPRA parse, Contract/Quote/Forward, catalog as-of, fail-loud validate. |
| pricing/ | Calculators plus the one fitted object. BSM catalog, IV invert, Engine (European analytic / American CRR), from_market, drift_check, calendar (session counts from the attested + upcoming calendars), daycount (vol-time convention as a passed object; day-bar default is the hybrid — sessions inside the calendar horizon, ACT/365 past it — stamped per row as bus/252 or act/365; money time is always ACT/365), term_structure (the ATM curve), surface (raw-SVI smile per expiry over the own-IV chain, with arb guards and term interpolation). |
| signals/ | Derived market series that sit on ingest + pricing reductions. signals.spot writes spy_spot — SPY cash level from the day-bar close, else the shortest-DTE parity proxy. signals.har_rv turns that series into rv_forecast — the walk-forward HAR-RV baseline with a lognormal uncertainty band. |
| tests/ | Offline pytest: ingest, tests/marketdata/, tests/pricing/, plus Healthchecks/schedule drift tests. |
| docs/ | This handbook: index.html plus one page per section, red-day.html / latent.html, and box-operations.html for off-repo machine state. Known holes that are in the tree live in IMPROVEMENTS.md. |
| deploy/ | schedule.json (source of schedule), crontab (kept installed through the timer cutover), the Ansible playbook + unit templates that generate the systemd timers, and an optional systemd unit for the websocket job. |
| scripts/ | bootstrap, backfill, backfill_underlying, repair, prune_raw, Healthchecks setup, build_term_structure.py (rebuilds the ATM curve over the archive), build_surface.py (same for the SVI surface), build_spy_spot.py (SPY spot series + overlap calibration), build_rv_forecast.py (walk-forward HAR-RV over the spot archive). Not a package. |
Not in the repo
There is no surfaces/, no rates package, no ClickHouse
client. The schemas module mentions ClickHouse in a docstring; no DDL
ships.