| outcomes | incorrect 22 · compile_fail 17 · correct 10 · error 10 · in_progress 5 |
| avg speedup | 2.07x (over 10 correct) |
| avg memory ratio (kernel / ref) | 1.34x (10 samples) |
| avg fusion ratio (ref launches / kernel launches) | 7.07x (10 samples) |
| avg energy ratio (ref / kernel) | — (0 samples) |
| avg SOL score (0–1) | 0.293 (8 samples) |
| avg DRAM util % | 16.6% |
| avg occupancy % | 76.1% |
| avg max abs error | 6.25e-03 |
| model | n | correct | compiled | avg speedup |
|---|---|---|---|---|
| FW-GLM-5-1 | 16 | 1 | 9 | 1.71x |
| Llama-4-Maverick-17B-128E-Instruct-FP8 | 16 | 0 | 0 | 0.00x |
| gpt-5.5-priority | 16 | 8 | 13 | 2.33x |
| grok-4.3 | 16 | 1 | 10 | 0.31x |