From the source
Subquadratic is the only company exclusively preoccupied with how foundation model algorithms scale with regards to compute, memory, and data.
- Most recent AI progress has come from scaling transformers with more parameters, data, and compute.
- We work on the scaling relationships underneath, with the goal of building the most compute, memory, and sample efficient algorithms and models.
- We started with the attention, whose compute cost grows quadratically with context length in a standard transformer.
- To address this challenge, earlier this year we announced Subquadratic Sparse Attention (SSA) , the architectural innovation that powers SubQ , the first fully subquadratic LLM, which has remained frontier-competitive through its updates.
- Today we're releasing SubQ Snap , a decision model for agentic workflows built on the same foundational research.
- Our decision model is built to take the data-intensive but structured decisions inside an agentic workflow, like choosing which files are relevant, which tool to use next, whether a test result passes, or which step needs a frontier model.
- …