Generated from build_system_prompt, build_problem_message, and the tool registry. Click any section to expand.
Conversation prompts
System prompt (default tool set)
You are an expert GPU kernel engineer. Write a custom CUDA kernel that replaces a PyTorch reference, producing numerically equivalent output faster than the reference. You have 10 turns and 30 tool calls. Each turn is one response from you, optionally with tool calls. `submit_kernel` records your final result and ends the session — call it once. ## Correctness Outputs must match the reference within tolerance: fp32 uses atol=rtol=1e-4; fp16/bf16 use atol=rtol=1e-2. Do not use try/except fallbacks to the reference, patch timing or RNG functions, or otherwise route around the evaluation — these are detected and fail the run. ## Tools Use the tools the way you would run an iteration loop by hand: write a kernel, compile it, check correctness, change something, repeat, submit. There is no required order and no required minimum number of iterations. The function-calling schema lists every tool with a short description of what it does and returns. Speedup vs the PyTorch reference and SOL (Speed of Light — the higher of DRAM and compute utilization, on a 0–1 scale) are the objectives. Other counters in tool output (occupancy, warp stalls, register count, instruction mix) are diagnostics — useful for figuring out *why* a kernel is slow, not goals in themselves.
System prompt (all tools, including profile / disassemble / ert)
You are an expert GPU kernel engineer. Write a custom CUDA kernel that replaces a PyTorch reference, producing numerically equivalent output faster than the reference. You have 10 turns and 30 tool calls. Each turn is one response from you, optionally with tool calls. `submit_kernel` records your final result and ends the session — call it once. ## Correctness Outputs must match the reference within tolerance: fp32 uses atol=rtol=1e-4; fp16/bf16 use atol=rtol=1e-2. Do not use try/except fallbacks to the reference, patch timing or RNG functions, or otherwise route around the evaluation — these are detected and fail the run. ## Tools Use the tools the way you would run an iteration loop by hand: write a kernel, compile it, check correctness, change something, repeat, submit. There is no required order and no required minimum number of iterations. You also have profiling and disassembly tools (`profile_kernel`, `disassemble_kernel`, `ert_roofline`) that report runtime hardware counters — DRAM bandwidth, warp stalls, occupancy, register usage, instruction mix — and an empirical roofline benchmark; each profiling call takes a few seconds to a few minutes. The function-calling schema lists every tool with a short description of what it does and returns. Speedup vs the PyTorch reference and SOL (Speed of Light — the higher of DRAM and compute utilization, on a 0–1 scale) are the objectives. Other counters in tool output (occupancy, warp stalls, register count, instruction mix) are diagnostics — useful for figuring out *why* a kernel is slow, not goals in themselves.
First user message (problem statement)
## Task Replace the forward computation of the PyTorch model below with a custom CUDA kernel. Your `ModelNew` class must: 1. Accept the same constructor arguments as `Model`. 2. Implement `forward()` with the same signature. 3. Produce numerically equivalent outputs at fp32 precision. 4. Run faster than the PyTorch reference. ## Output format Your submission is a complete Python file that defines a class called `ModelNew`. The file is executed with `exec()` before we instantiate `ModelNew`, so any module-level setup runs first. Use `torch.utils.cpp_extension.load_inline` to compile and bind CUDA source at module load time, then call the compiled extension from `ModelNew.forward`. Do not submit raw CUDA C or a standalone .cu file. Do not write your own `PYBIND11_MODULE(...)` block in `cpp_sources`. `load_inline` auto-generates one from the `functions=[...]` argument; including your own causes a duplicate-symbol redefinition error at compile time. List the host-side wrapper function names in `functions=[...]` instead. ## Reference implementation ```python <reference PyTorch module source goes here> ```
Turn-warning message (injected when budget runs low)
Early — multiple turns still left (sample: 2 turns / 5 tool calls remaining)
Status: 2 turns and 5 tool calls remain. If you have a correct kernel and no concrete next change you want to try, submit now.
Last turn (sample: 1 turn / 1 tool call remaining)
Status: 1 turn and 1 tool call remain. Last turn — call `submit_kernel` with your best correct kernel.
Tool descriptions
Each entry below is one function declaration sent to the model. default means the tool is included when tools = "default" in the sweep TOML.
compile_kernel default
Compile the kernel without running it. Use this first after writing or editing a kernel to catch syntax, linker, and CUDA-compilation errors cheaply before spending GPU time on correctness. Returns compiler output on failure, or a success confirmation.
input_schema
{
"type": "object",
"properties": {
"kernel_code": {
"type": "string",
"description": "Full Python source of the ModelNew kernel file. Must be a complete, valid Python module \u2014 not raw CUDA C/C++."
}
},
"required": [
"kernel_code"
]
}run_correctness default
Run the kernel against the reference for correctness only — no timing. Use this after compile_kernel succeeds, to verify your kernel produces numerically equivalent outputs. Returns per-trial pass/fail status and the nature of any numerical or runtime errors.
input_schema
{
"type": "object",
"properties": {
"kernel_code": {
"type": "string",
"description": "Full Python source of the ModelNew kernel file. Must be a complete, valid Python module \u2014 not raw CUDA C/C++."
}
},
"required": [
"kernel_code"
]
}profile_kernel
Profile the kernel with NVIDIA Nsight Compute. Returns comprehensive diagnostics: DRAM bandwidth utilization, compute throughput (FP32/FP16/tensor-core), per-kernel breakdown, warp stall reasons, memory coalescing quality, shared-memory bank conflicts, L1/L2 hit rates, occupancy with limiting factors (registers, smem, block size), pipe utilization, branch divergence, eligible-warps analysis, and targeted data-driven optimization hints. Supports delta comparison across iterations. Use when you have a correct kernel and need to understand why it is slow.
Multi-rank mode: when world_size>1 (8xH100 sweeps), profiling runs ncu --target-processes all around a torchrun --nproc_per_node=N launch and reports a full ProfileSummary for EACH rank — load imbalance, divergent stalls, and per-rank occupancy are all visible. Delta vs previous iteration is computed per rank. Profiling is fail-hard: any missing rank report or any rank with zero captured metrics aborts the call.
input_schema
{
"type": "object",
"properties": {
"kernel_code": {
"type": "string",
"description": "Full Python source of the ModelNew kernel file. Must be a complete, valid Python module \u2014 not raw CUDA C/C++."
}
},
"required": [
"kernel_code"
]
}get_gpu_specs default
Return peak hardware specs for the GPU this kernel will run on (memory bandwidth, TFLOPS per precision, SM count, shared memory per SM, register file size, etc.). Use this once at the start to calibrate your optimization targets.
input_schema
{
"type": "object",
"properties": {},
"required": []
}static_check default
Run a static-analysis pass that detects reward-hacking patterns (try/except fallbacks to the reference, timing-function patches, lazy-tensor tricks, threading injection, etc.). Use this before submit_kernel as a sanity check — flagged submissions cause evaluation to fail.
input_schema
{
"type": "object",
"properties": {
"kernel_code": {
"type": "string",
"description": "Full Python source of the ModelNew kernel file. Must be a complete, valid Python module \u2014 not raw CUDA C/C++."
}
},
"required": [
"kernel_code"
]
}submit_kernel default
Submit the final kernel for full evaluation: correctness check AND timing measurement. This ends the session — only call it when you are confident the kernel is correct and optimized. Returns kernel runtime in microseconds. The reference runtime and speedup ratio are NOT revealed.
input_schema
{
"type": "object",
"properties": {
"kernel_code": {
"type": "string",
"description": "Full Python source of the ModelNew kernel file. Must be a complete, valid Python module \u2014 not raw CUDA C/C++."
}
},
"required": [
"kernel_code"
]
}disassemble_kernel
Disassemble the compiled CUDA kernel to inspect its native GPU assembly (SASS), PTX intermediate representation, and per-kernel resource usage (registers, shared memory, spills). Use this when you have a correct kernel and want to understand the compiler's code generation — register pressure, instruction mix (memory vs compute vs control), tensor-core usage, and register spills. Requires cuobjdump and nvdisasm (shipped with CUDA Toolkit).
input_schema
{
"type": "object",
"properties": {
"kernel_code": {
"type": "string",
"description": "Full Python source of the ModelNew kernel file. Must be a complete, valid Python module \u2014 not raw CUDA C/C++."
}
},
"required": [
"kernel_code"
]
}ert_roofline
Run the Empirical Roofline Tool — micro-benchmarks that measure the actual (not theoretical) peak memory bandwidth and compute throughput of the current GPU. Returns measured bandwidth at each memory hierarchy level (L1, L2, DRAM/HBM) and peak TFLOPS (FP32, FP16/tensor-core). Also computes the ridge point — the arithmetic intensity where the roofline transitions from memory- to compute-bound. Results are cached per GPU so subsequent calls are instant. No arguments needed.
input_schema
{
"type": "object",
"properties": {},
"required": []
}