Tabular

Finally, let’s take a look at fastai.tabular models. (We don’t need to look at collaborative filtering separately, since we’ve already seen that these models are just tabular models, or use the dot product approach, which we’ve implemented earlier from scratch.)

Here is the forward method for TabularModel:

if self.n_emb != 0:
    x = [e(x_cat[:,i]) for i,e in enumerate(self.embeds)]
    x = torch.cat(x, 1)
    x = self.emb_drop(x)
if self.n_cont != 0:
    x_cont = self.bn_cont(x_cont)
    x = torch.cat([x, x_cont], 1) if self.n_emb != 0 else x_cont
return self.layers(x)

We won’t show __init__ here, since it’s not that interesting, but we will look at each line of code in forward in turn. The first line:

if self.n_emb != 0:

is just testing whether there are any embeddings to deal with—we can skip this section if we only have continuous variables. self.embeds contains the embedding matrices, so this gets the activations of each:

x = [e(x_cat[:,i]) for i,e in enumerate(self.embeds)]

and concatenates them into a single tensor:

x = torch.cat(x, 1)

Then dropout is applied. You can pass embd_p to __init__ to change this value:

x = self.emb_drop(x)

Now we test whether there are any continuous variables to deal with:

if self.n_cont != 0:

They are passed through a batchnorm layer:

x_cont = self.bn_cont(x_cont)

and concatenated with the embedding activations, if there were any:

x = torch.cat([x, x_cont], 1) if self.n_emb != 0 else x_cont

Finally, this is passed through the linear layers (each of which includes batchnorm, if use_bn is True, and dropout, if ps is set to some value or list of values):

return self.layers(x)

Congratulations! Now you know every single piece of the architectures used in the fastai library!