What is a Transformer?

The Transformer(Technology) is a neural-network architecture introduced in 2017.

Self-attention

Each token looks at every other token in the input and decides how much to weight it. That single mechanism replaced the recurrent networks that came before.

Why it scales

Because self-attention is parallel, transformers train efficiently on modern GPU(Hardware)s. That is what made Large Language Model(Technology)s possible.

Explore Further

Entities mentioned

Transformer(Technology)Neural Network(Technology)Large Language Model(Technology)GPU(Technology)