AI / ML

Show HN: Automatic deduction of FlashAttention kernel from ~25 lines of SQL

The author demonstrates a SQL query that automatically deduces a FlashAttention kernel, a type of neural network component, using three portable SQL extensions: xarray-sql, ddx, and einfold. These extensions enable efficient representation of large datasets and automatic differentiation in SQL. The author suggests that this approach provides a fresh perspective on accelerator programming and separates logical and physical layers in tensor programs, making them more correct and scalable.

Read the full article at github.com →