
GPU kernel optimization for CUDA developers: AutoKernel
, 3 min read, By Jaber Jaber
EngineeringAutoKernel is RightNow AI's open-source tool for GPU kernel optimization. AutoKernel profiles PyTorch models, tests Triton variants, and follows measured bottlenecks. A coding agent edits one kernel at a time, runs a fixed benchmark, and keeps or reverts each change.
What does AutoKernel do?
Give AutoKernel any PyTorch model. It profiles the model to find which GPU kernels are bottlenecks. It extracts each bottleneck as a standalone Triton or CUDA kernel. It optimizes each kernel autonomously: edit, benchmark, keep or revert, and repeat. It verifies end-to-end correctness and reports the total speedup.

How does the loop work?
The agent reads program.md, which contains comprehensive instructions for autonomous operation. It includes a 6-tier optimization playbook, a decision framework, crash handling and Amdahl's law reasoning.
The agent edits kernel.py one kernel at a time. It runs bench.py, a fixed benchmark with 5-stage correctness checks and roofline analysis, and either keeps or reverts the change.
The orchestrator decides when to move to the next kernel using Amdahl's law. It prioritizes by impact and moves on when diminishing returns set in. The agent only touches kernel.py, so scope stays manageable, diffs reviewable and reverts clean.
Every experiment is logged to a plain results.tsv file. Each row records the kernel, its throughput, its latency and whether it passed: PASS, FAIL, TIMEOUT or CRASH.
How does it keep a kernel correct?
The benchmark checks kernel output against PyTorch before measuring performance. A fast but wrong kernel is immediately reverted. The five correctness stages are smoke, shape sweep, numerical stability, determinism and edge cases.
Each kernel type has a PyTorch reference in reference.py. At the end, verify.py plugs the optimized kernels back into the model, checks end-to-end correctness and reports the total speedup.
How do you start?
Clone the repository and run uv sync. Run uv run prepare.py once to set up test data and baselines. Profile a model with profile.py, extract the top bottleneck kernels with extract.py, and check that the benchmark works with bench.py.
Then start a coding agent in the directory and ask it to read program.md and start with setup. The agent presents the optimization plan, creates a branch and optimizes each bottleneck kernel in priority order. program.md is written so the agent can run for 10 hours or more without getting stuck.
Need CUDA set up? See The Complete Guide to Fixing CUDA Installation Issues.
Where is the research behind it?
RightNow AI is a YC-backed GPU research lab. The paper is AutoKernel: Autonomous GPU Kernel Optimization via Iterative Agent-Driven Search, by Jaber Jaber and Osama Jaber, 2026. It is on arXiv as 2603.21331, and its code is the AutoKernel repository.
The lab's AutoMegaKernel paper, A Statically-Checked Agent Harness for Self-Retargeting Megakernel Synthesis, is on arXiv as 2606.09682. See RightNow Research Lab for every paper and open-source project.
Short answers
Is AutoKernel open source?
Yes. Its license is MIT, and the code is at github.com/RightNow-AI/autokernel.
Which models can it profile?
Any PyTorch model: profile.py profiles it with torch.profiler, ranks kernels by GPU time and classifies each as compute-bound or memory-bound. Self-contained model definitions ship with it, with no transformers library needed.
Which kernels does it cover?
Its supported kernels include matmul, softmax, layernorm, rmsnorm, flash_attention, fused_mlp, cross_entropy, rotary_embedding and reduce.
Does it write Triton or CUDA?
Both. Triton is for fast iteration, and CUDA gives direct access to tensor cores. Both backends share the same kernel_fn() interface.
Which GPU does it need?
An NVIDIA GPU, with Python 3.10 or later and uv.