AI AI Blazers

Reading Route

Road to Large Language Models

From Seq2Seq and Attention to Transformer, BERT, GPT, scaling laws, instruction tuning, and RLHF.

1
2014 · NLP · Seq2Seq
Sequence to Sequence Learning with Neural Networks
Read now
2
2014 · NLP · Attention
Neural Machine Translation by Jointly Learning to Align and Translate
Read now
3
2017 · NLP · Transformer
Attention Is All You Need
Read now
4
2018 · NLP · BERT
BERT: Pre-training of Deep Bidirectional Transformers
Read now
5
2018 · NLP · GPT-1
Improving Language Understanding by Generative Pre-Training
Read now
6
2020 · LLM · Scaling Laws
Scaling Laws for Neural Language Models
Coming soon
7
2020 · LLM · GPT-3
Language Models are Few-Shot Learners
Coming soon
8
2021 · LLM · Instruction Tuning
Finetuned Language Models Are Zero-Shot Learners
Coming soon
9
2022 · LLM · RLHF
Training Language Models to Follow Instructions with Human Feedback
Coming soon
10
2025 · LLM · RL for NLU
Improving the Language Understanding Capabilities of Large Language Models Using Reinforcement Learning
Read now