Documented knobs
From .env.example (no secrets). Process environment overrides
the file. Do not commit .env. Where each value actually lives
on the box: box operations.
Credentials and endpoints
| Name | Role |
|---|---|
| MASSIVE_API_KEY | Required. Query-param auth. |
| MASSIVE_API_BASE | Default https://api.polygon.io. |
| WS_DELAYED_URL | Default wss://delayed.massive.com/options. |
| MASSIVE_S3_ACCESS_KEY_ID MASSIVE_S3_SECRET_ACCESS_KEY | Flat-file access keys (placeholders exit 3 on pull). |
| MASSIVE_S3_ENDPOINT MASSIVE_S3_BUCKET | Flat-file endpoint and bucket (defaults https://files.massive.com / flatfiles). |
| IBKR_FLEX_TOKEN IBKR_FLEX_QUERY_ID IBKR_ACCOUNT_ID | Read-only Flex Web Service (no gateway). Commented in .env.example; required only for ibkr_executions. Token expires ~yearly. |
Paths
| Name | Role |
|---|---|
| DATA_ROOT | Warehouse root. Default /data/massive. |
| LOG_ROOT | Logs. Default DATA_ROOT/logs (not the warehouse root). Settings.load in ingest/common/config.py. |
Healthchecks and canary
| Name | Role |
|---|---|
| HEALTHCHECKS_PING_KEY | Per-job checks. Without it, a dead job is silent. |
| HEALTHCHECKS_API_KEY | Management key used only by scripts/setup_healthchecks.py to create/update the checks. Not read at run time, and not the ping key. |
| HEALTHCHECKS_BASE | Ping root (hosted https://hc-ping.com; self-host https://<host>/ping). |
| DRIFT_CHECK_R | Optional scalar r override for the 17:00 canary. Unset → landed Treasury curve per contract. 0.04 is only the fallback when rates_sync has never run. |
| DRIFT_ALERT_WEBHOOK | Optional FAIL POST. Healthchecks + exit 1 remain the alert. |
Code-side knobs
Also in code, not in .env.example:
MASSIVE_MAX_RPS (60 — sustained; probed clean to 100 rps
on 2026-09-03) and MASSIVE_BURST (40 — held back
deliberately, because the 429s seen on 2026-08-31 came from tight
bursts rather than sustained pacing),
MASSIVE_RATELIMIT_SHARED (1 — the token bucket is
cross-process by default, so every job on the box draws on one
allowance rather than one each; set 0 for the old per-process
behaviour),
MASSIVE_RATELIMIT_STATE
(DATA_ROOT/_meta/ratelimit.json),
MASSIVE_PRIORITY_RESERVE (0.35 — the fraction of the
bucket reserved for snapshot_sweep; every other capture
job draws at low priority and stops at this floor, which is where
“everything else yields API budget to it” is actually
enforced),
MASSIVE_PRIORITY_CLAIM_S (1.0 — a waiting NORMAL caller
stamps this so LOW callers stand aside),
TRADES_CONCURRENCY (8),
TRADES_BACKOFF_MAX (4 — adaptive poll slots; set 1 to disable),
RETAIN_DAYS (90),
QUARANTINE_RETAIN_DAYS (30 — superseded re-filter output
under _quarantine/, pruned on the monthly run),
FLATFILE_RETAIN_DAYS (365 — only consulted with
prune_raw.sh --flatfiles, which is opt-in),
MIN_FREE_GB (100 on backfill),
TZ_NAME (America/New_York),
JOB_MAX_ATTEMPTS (3) / JOB_RETRY_BASE_S (30s,
doubling, capped at 300s) — in-run retry, since cron has no backoff.
Legacy HEALTHCHECKS_PING_URL is rejected.