error
7_Matmul_with_small_K_dimension_.py
started
2026-05-02T21:47:10Z
reference PyTorch source (the prompt the agent sees)
import torch
import torch.nn as nn
class Model(nn.Module):
"""
Simple model that performs a single matrix multiplication (C = A * B) with a small K dimension
"""
def __init__(self):
super(Model, self).__init__()
def forward(self, A: torch.Tensor, B: torch.Tensor) -> torch.Tensor:
"""
Performs matrix multiplication.
Args:
A: Input tensor of shape (M, K).
B: Input tensor of shape (K, N).
Returns:
Output tensor of shape (M, N).
"""
return torch.matmul(A, B)
M = 16384 * 2
N = 16384 * 2
K = 32 * 2
def get_inputs():
A = torch.rand(M, K)
B = torch.rand(K, N)
return [A, B]
def get_init_inputs():
return [] # No special initialization inputs neededturn 010.9s turn · 10.9s LLM · 0.0s tools
reasoning
Let me analyze this task. I need to write a custom CUDA kernel that performs matrix multiplication C = A * B where:
- A is (M, K) = (32768, 64)
- B is (K, N) = (64, 32768)
This is a very wide and tall matrix multiplication with a small K dimension (64).
For small K GEMMs, there are several optimizations to consider:
1. Since K=64 is small, we want to load A and B efficiently into shared memory
2. Since M and N are very large, we have plenty of parallelism
3. A standard cuBLAS approach might not be optimal for such a shape
Let me first get the GPU specs to understand what hardware I'm targeting.
call → get_gpu_specs
callget_gpu_specs
{}
result ← get_gpu_specs (ok)
okget_gpu_specs
GPU specs for NVIDIA H100 80GB HBM3:
total memory (runtime): 79.1 GB
spec entry: H100
GPU Architecture: Hopper
GPU Memory: 80GB
Memory Bandwidth: 3.35 TB/s
FP64 TFLOPS: 34
FP64 Tensor Core TFLOPS: 67
FP32 TFLOPS: 67
TF32 Tensor Core TFLOPS: 989 with sparsity
BFLOAT16 Tensore Core TFLOPS: 1979 with sparsity
FP16 Tensor Core TFLOPS: 1979 with sparsity
FP8 Tensor Core TFLOPS: 3958 with sparsity
INT8 Tensor Core TOPS: 3958 with sparsity
Register File Size: 64K 32-bit registers per SM
Maximum number of registers per thread: 255
Maximum number of thread blocks per SM: 32
Shared memory capacity per SM: 228 KB
Maximum shared memory per thread block: 227 KB
turn 17189.1s turn · 7189.1s LLM · 0.0s tools
feedbackLLM call failed: _WorkItemTimeout: work item exceeded soft timeout budget