Proposal

Target architecture

not built
end-to-end loop

What a systematic volatility trading system looks like end to end: data → forecasting → strategy → portfolio risk → execution → reconciliation → monitoring → research feedback.

This is a target, not a description of the box. Most of the right-hand side does not exist. There is no order manager, no broker adapter, no ledger, no reconciliation, and no portfolio risk engine — see Not built. What runs today is the left and middle: ingest, the warehouse, and the pricing / surface analytics described in Ingest and pricing.

The whole loop

Market data ingestion, warehousing, forecasting, strategy, risk, execution, reconciliation, monitoring, and research feedback loops INPUTS DATA FOUNDATION ANALYTICS / DECISION TRADING CONTROL LOOP RESEARCH / GOVERNANCE LOOP Live market data options quotes, trades, NBBO underlying, vols, rates, calendars real-time feed for execution and live risk Delayed / research feeds Massive delayed WS / REST development, lagged research, sanity checks Broker / account inputs orders, fills, positions, cash margin, commissions, statements IBKR API + Flex / statements Ingestion / capture timers, collectors, source adapters typed as-of reads + availability time immutable raw events, source metadata Validation / QC gate staleness, crossed markets, bad prints chain completeness, timestamp alignment block downstream actions if inputs are dirty Warehouse / event store raw / clean / canonical / decision logs parquet, dt partitions do not overwrite decision history Reference data contracts, multipliers, expiries rates, dividends, calendars, holidays instrument master + conventions Feature engineering RV, term structure, skew, liquidity event flags, regimes, macro context point-in-time feature generation Forecasting HAR-RV / baseline + uncertainty horizon-matched realized vol forecasts a distribution, not just a point estimate Pricing / surface analytics IV, greeks, forwards, carry BSM / CRR / SVI surface and repricing current market valuation, not the forecast Signal / strategy engine candidate spreads and structures entry, exit, hedge, roll, no-trade rules expected edge net of costs and slippage Portfolio construction / risk size proposals, concentration limits portfolio greeks, stress / scenario loss drawdown guardrails, margin buffer approve, resize, or reject each trade Order management / execution limit price policy, multi-leg handling partial fills, cancel/replace, retry logic current quotes required before sending orders Portfolio state / ledger positions, orders, fills, fees, cash match internal book to broker state accounting truth and replay truth Monitoring / kill switches data freshness, unknown order states risk breaches, model degradation, alerts block new risk, cancel orders, controlled flatten Backtesting / event replay same strategy and same risk logic bid/ask, fees, no-fill and partial-fill realism walk-forward only, point-in-time inputs Evaluation / attribution forecast error, calibration, regime analysis P&L by delta / vega / theta / costs did the model add edge, or just risk? Research sandbox new features, structures, and filters paper trading, simulation, stress replay promotion only after evidence Governance / audit / dashboards decision logs, configs, model versions, approvals, risk snapshots daily dashboards, run reports, exception logs, post-trade reviews reproducible from market input to broker-confirmed outcome fills / positions / account state halt, reject, reduce risk continuous improvement / promotion after validation
market / broker inputs data, storage, QC modeling, pricing, decision execution, state, control research / governance

What it adds

Implementation order

  1. Canonical event store, reference data, QC gates.
  2. Pricing, feature generation, baseline RV forecast.
  3. One defined-risk strategy engine with explicit exits.
  4. Portfolio risk engine with hard guardrails and stress tests.
  5. OMS / execution simulator, then the broker adapter.
  6. Ledger, reconciliation, dashboards, alerts.
  7. Event replay and backtesting that run the same production logic.

Operating principles

  1. Keep delayed research data separate from current execution data.
  2. Forecast future realized volatility; do not confuse it with fitting the current implied vol surface.
  3. Every trade passes portfolio-level risk checks before execution.
  4. Preserve immutable decision logs and reconcile continuously against broker reality.