In [ ]:

#hide
!pip install -Uqq fastbook
import fastbook
fastbook.setup_book()

In [ ]:

#hide
from fastbook import *

[[chapter_nlp_dive]]

A Language Model from Scratch

We’re now ready to go deep… deep into deep learning! You already learned how to train a basic neural network, but how do you go from there to creating state-of-the-art models? In this part of the book we’re going to uncover all of the mysteries, starting with language models.

You saw in <> how to fine-tune a pretrained language model to build a text classifier. In this chapter, we will explain to you what exactly is inside that model, and what an RNN is. First, let’s gather some data that will allow us to quickly prototype our various models.