recallkit measures how far back a sequence model can actually recall, finds the true attention-window boundary of any released checkpoint, and computes the smallest window that meets your recall target — before you spend a dollar on training.
Generates recall probes whose retrieval distance is exact by construction — separated from context length, which every needle-in-a-haystack benchmark confounds.
Finds a released checkpoint's effective window and boundary convention from behaviour alone — not from its config file, which is wrong more often than you'd think.
Feeds your workload's measured distance distribution into the law and returns the smallest window — and cheapest model shape — that hits your recall target.
Every prediction below was committed to a public, timestamped ledger before the models were scored.
| window | predicted recall | observed | error |
|---|---|---|---|
| W = 128 | 0.2615 | 0.2627 | 0.001 |
| W = 256 | 0.4769 | 0.4775 | 0.001 |
| W = 512 | 0.7231 | 0.7251 | 0.002 |
| W = 1024 | 1.0000 | 0.9951 | 0.005 |
110M-parameter models trained on natural text, four window sizes, all six pre-registered gates passed. An eight-cell out-of-sample test across three unseen task geometries matched within 0.005 of prediction — every cell. recallkit's provision reproduces the sealed predictions to float-exact zeros in its test suite.
The service version: send us your workload's structure — content never leaves your side; the telemetry instrument records token distances, not text — and we return the smallest model configuration that will hit your recall target, with the prediction in writing before any conversion is paid for.
pip install recallkit-ai — available when the preprint lands. Apache-2.0 with an express patent grant.