Fast Transformer
Low-level transformer optimization that reached 1,152.3 tokens per second.
A performance-focused entry for Dryft's Hack the North challenge. The project experiments with transformer kernels, memory layouts, fused operations, and autoresearch tooling to make autoregressive inference generate tokens as quickly as possible.
Highlights
- 6th place out of 50+ teams in Dryft's Hack the North challenge
- Measured 1,152.3 tokens per second on the challenge benchmark
- Optimization workflow spanning custom engine code, tests, and benchmark tooling
- Autoresearch agent setup for running and comparing performance experiments
Technology
Python, PyTorch, Triton, CUDA, Benchmarking