Crack the Sky
marketdata
Typed parquet I/O. Import submodules directly
(marketdata.opra, .types, .catalog,
.validate). CLI: python -m marketdata.validate
(same as python -m marketdata).
- OPRA allowlist
-
Allowlist is exactly
SPY,SPX,SPXW,VIX,VIXW. The parser alternation is longest-first (SPXWbeforeSPX,VIXWbeforeVIX) soO:SPXW…is not captured asSPXandO:VIXW…is notVIX. A prefix match onO:SPYwould admit SPYL/SPYS/SPYG, andO:VIXwould admit VIXY;parse_opradoes not. RaisesOPRAParseError— never a partial. - Contract
- Exercise style and multiplier live on the dataclass: SPY American, SPX/SPXW/VIX/VIXW European, multiplier 100. Settlement instants: SPY/SPXW 16:00 ET (PM-settled); SPX monthlies and both VIX series 09:30 ET (AM-settled to the Wednesday SOQ).
- Quote
-
Snapshot-derived. Vendor IV/greeks copied as diagnostics
(
vendor_*).market_price= last if present, else day close; the trade's own stamp rides along aslast_trade_asof_nsso consumers can age-filter (asof_nsstays the underlying/snapshot clock). Pricing must ignore vendor columns. - As-of
-
Snapshots/contracts/forwards are written many times a day.
read_partitionrefuses those datasets (double-count).read_asoftakes the last file per underlying at or beforeasof_ns(filename epoch-ms). Absence is an error.option_tradesis multi-source: a partition can hold overlapping REST and flat-file rows, so readers must passsrc.decision_logis the opposite case — append-only event history, soread_asofrefuses it (last-file-wins would hide earlier events); read the whole partition viaingest.common.decision_log.read_decisions. - Fail-loud
-
Extra or missing parquet columns →
SchemaError. Validate fails on foreign roots, mixed underlyings vs a SPY-only query, required-nulls, empty allowlisted root, empty table.--rootsnarrows the allowlist; it cannot extend it.