resolve rate by difficulty
resolve rate by repo
failure labels (failed trials only)
agreement matrix (tasks solved by k models)
steps per trial
cost per trial (USD)
| task | repo | difficulty | model | result | label | exit | steps | out tok | cost | sec | loc | tests run | f2p ✓ | f2p ✗ | p2p ✗ |
|---|