| outcomes | compile_fail 23 · incorrect 18 · correct 13 · error 10 |
| avg speedup | 2.32x (over 13 correct) |
| avg memory ratio (kernel / ref) | 0.79x (13 samples) |
| avg fusion ratio (ref launches / kernel launches) | 8.65x (13 samples) |
| avg energy ratio (ref / kernel) | — (0 samples) |
| avg SOL score (0–1) | 0.263 (11 samples) |
| avg DRAM util % | 15.7% |
| avg occupancy % | 67.8% |
| avg max abs error | 6.20e-03 |
| model | n | correct | compiled | avg speedup |
|---|---|---|---|---|
| FW-GLM-5-1 | 16 | 3 | 12 | 2.98x |
| Llama-4-Maverick-17B-128E-Instruct-FP8 | 16 | 0 | 0 | 0.00x |
| gpt-5.5-priority | 16 | 10 | 13 | 2.12x |
| grok-4.3 | 16 | 0 | 6 | 0.00x |