| outcomes | compile_fail 25 · in_progress 19 · correct 10 · incorrect 5 · error 2 |
| avg speedup | 4.92x (over 10 correct) |
| avg memory ratio (kernel / ref) | 0.85x (10 samples) |
| avg fusion ratio (ref launches / kernel launches) | 2.75x (10 samples) |
| avg energy ratio (ref / kernel) | 1.60x (9 samples) |
| avg SOL score (0–1) | 0.720 (10 samples) |
| avg DRAM util % | 75.1% |
| avg occupancy % | 209.3% |
| avg max abs error | 9.06e-04 |
| model | n | correct | compiled | avg speedup |
|---|---|---|---|---|
| FW-GLM-5-1 | 7 | 2 | 4 | 0.66x |
| Llama-4-Maverick-17B-128E-Instruct-FP8 | 18 | 1 | 1 | 0.84x |
| gpt-5.5-priority | 19 | 4 | 7 | 10.60x |
| grok-4.3 | 17 | 3 | 3 | 1.56x |