State Space Model

What is a state space model?

When processing a sequence — say a sentence — a Transformer lets every word look at every other word. That's information-rich, but the cost grows quadratically with length. A state space model (SSM) takes a different route: like a slow-moving river, it carries a compressed "hidden state" forward, updating it as it reads each item. The result: less memory and time on long sequences.

How is it different from a Transformer?

Transformer: everyone looks at everyone
Every position attends to every other — great results, but pricier as sequences grow.
SSM: a relay of state
It leans on a single state passed along, which shines on very long text.

Why is it suddenly everywhere?

Linear complexity
Double the length, roughly double the cost — not quadruple — so long documents are cheap.
Long-range memory
Even when relevant context sits far away, the SSM can hold onto it.
Hardware-friendly
The compute pattern is regular, which runs efficiently on GPUs.

Its stars and limits

Mamba is one of the best-known SSM variants. SSMs show real promise for long sequences and real-time inference, though Transformers still lead on tasks that need fine-grained global comparison. The future likely isn't either/or — it's both.

Bottom line: state space models are a new route through sequences — a compressed relay of state that buys efficiency on long text.