TECHNOLOGY

Transformer

The Transformer is a neural-network architecture introduced in 2017. Its self-attention mechanism became the foundation for modern large language models.

Introduced: 2017 (Vaswani et al.)
Paper: Attention Is All You Need

TECHNOLOGYTransformer

Trails Articles

Key facts

What we know

Introduced: 2017 (Vaswani et al.)
Paper: Attention Is All You Need

Forest Trails

Connections in the forest

Referencing articles

5 articles

Technology9 minUpdated May 20, 2025

The rise of large language models

An LLM is a Transformer trained on a lot of text. The interesting behaviour appears only above a certain scale.

Technology7 minUpdated Apr 8, 2025

What is a Transformer?

The Transformer replaced recurrence with attention — and unlocked models that scale with data and compute in ways nothing else did.

Technology8 minUpdated Mar 11, 2025

What is a neural network?

A neural network is a stack of simple functions with lots of adjustable knobs. Training tunes the knobs. That is the whole idea.

Companies7 minUpdated Mar 5, 2025

Inside Google

Google began as a search engine. Its infrastructure — and its ambitions — grew from there.

Technology7 minUpdated Feb 18, 2025

Understanding machine learning

Machine learning is not magic. It is a set of methods for turning examples into decisions. Here is the underlying idea.

Continue exploring

Neighboring entities in the forest.

Amazon(Company)Amazon Web Services(Product)AMD(Company)Andrej Karpathy(Person)Anthropic(Company)Apple(Company)Apple Silicon(Product)Artemis Program(Concept)

Or search the whole forest →