error
13_QuantVLA_SelectiveLayoutFusion.py
started
2026-05-04T06:01:06Z
reference PyTorch source (the prompt the agent sees)
import torch
import torch.nn as nn
# Default ModelNew is PyTorch-only. run_and_check: use check_kernel=False with backend=cuda
# until the candidate file contains a real CUDA kernel (see kernel_static_checker).
def _fake_quant_symmetric_int8(x: torch.Tensor, scale: torch.Tensor) -> torch.Tensor:
"""Symmetric int8 fake-quant: round(x/scale) clamped to [-127, 127], times scale."""
q = torch.clamp(torch.round(x / scale), -127.0, 127.0)
return q * scale
class Model(nn.Module):
"""
**QuantVLA selective layout fusion** (reference target for a fused CUDA/Triton kernel).
Vision–language–action stacks often use **token-wise mixed precision**: some
timesteps or modalities tolerate aggressive quantization while others stay in
full precision. This module fuses in **one forward**:
1. **Layout routing** — ``sigmoid(layout_logits)`` yields a per-token mix
coefficient in ``(0, 1)`` (quant vs FP residual).
2. **Quantitative path** — global symmetric **int8-style fake quantization**
(scale from the tensor max, ``/ 127``).
3. **Selective blend** — ``α · x_quant + (1 - α) · x``.
4. **Head** — linear projection on the mixed activations.
This matches a “quantitative selective layout” block: the layout logits (from a
router, modality tags, or calibration) choose how strongly each position uses the
quantized layout vs the identity layout.
"""
def __init__(self, dim: int):
super().__init__()
self.dim = dim
self.out_proj = nn.Linear(dim, dim, bias=True)
def forward(self, x: torch.Tensor, layout_logits: torch.Tensor) -> torch.Tensor:
"""
Args:
x: (B, T, D) VLA token activations (fused modalities).
layout_logits: (B, T) unbounded logits; larger → more mass on quantized path.
Returns:
(B, T, D) updated activations.
"""
B, T, D = x.shape
assert D == self.dim
assert layout_logits.shape == (B, T)
alpha = torch.sigmoid(layout_logits).unsqueeze(-1)
with torch.no_grad():
amax = x.detach().abs().max().clamp(min=1e-8)
scale = amax / 127.0
x_q = _fake_quant_symmetric_int8(x, scale)
mixed = alpha * x_q + (1.0 - alpha) * x
return self.out_proj(mixed)
class ModelNew(nn.Module):
"""KernelBench candidate entry point; default delegates to ``Model``."""
def __init__(self, dim: int):
super().__init__()
self._impl = Model(dim)
def forward(self, x: torch.Tensor, layout_logits: torch.Tensor) -> torch.Tensor:
return self._impl(x, layout_logits)
batch_size = 8
seq_len = 64
dim = 128
def get_inputs():
p = popcorn_pri
x = torch.randn(p.jitter_int(batch_size), p.jitter_int(seq_len), dim)
layout_logits = torch.randn(p.jitter_int(batch_size), p.jitter_int(seq_len))
return [x, layout_logits]
def get_init_inputs():
return [dim]
turn 010.9s turn · 10.9s LLM · 0.0s tools
call → get_gpu_specs
callget_gpu_specs
{}
result ← get_gpu_specs (ok)
okget_gpu_specs
GPU specs for NVIDIA H100 80GB HBM3:
total memory (runtime): 79.1 GB
spec entry: H100
GPU Architecture: Hopper
GPU Memory: 80GB
Memory Bandwidth: 3.35 TB/s
FP64 TFLOPS: 34
FP64 Tensor Core TFLOPS: 67
FP32 TFLOPS: 67
TF32 Tensor Core TFLOPS: 989 with sparsity
BFLOAT16 Tensore Core TFLOPS: 1979 with sparsity
FP16 Tensor Core TFLOPS: 1979 with sparsity
FP8 Tensor Core TFLOPS: 3958 with sparsity
INT8 Tensor Core TOPS: 3958 with sparsity
Register File Size: 64K 32-bit registers per SM
Maximum number of registers per thread: 255
Maximum number of thread blocks per SM: 32
Shared memory capacity per SM: 228 KB
Maximum shared memory per thread block: 227 KB
turn 15342.2s turn · 5342.2s LLM · 0.0s tools
feedbackLLM call failed: _WorkItemTimeout: work item exceeded soft timeout budget