Questionnaire

  1. What is the “head” of a neural net?
  2. What is the “body” of a neural net?
  3. What is “cutting” a neural net? Why do we need to do this for transfer learning?
  4. What is model_meta? Try printing it to see what’s inside.
  5. Read the source code for create_head and make sure you understand what each line does.
  6. Look at the output of create_head and make sure you understand why each layer is there, and how the create_head source created it.
  7. Figure out how to change the dropout, layer size, and number of layers created by cnn_learner, and see if you can find values that result in better accuracy from the pet recognizer.
  8. What does AdaptiveConcatPool2d do?
  9. What is “nearest neighbor interpolation”? How can it be used to upsample convolutional activations?
  10. What is a “transposed convolution”? What is another name for it?
  11. Create a conv layer with transpose=True and apply it to an image. Check the output shape.
  12. Draw the U-Net architecture.
  13. What is “BPTT for Text Classification” (BPT3C)?
  14. How do we handle different length sequences in BPT3C?
  15. Try to run each line of TabularModel.forward separately, one line per cell, in a notebook, and look at the input and output shapes at each step.
  16. How is self.layers defined in TabularModel?
  17. What are the five steps for preventing over-fitting?
  18. Why don’t we reduce architecture complexity before trying other approaches to preventing overfitting?

Further Research

  1. Write your own custom head and try training the pet recognizer with it. See if you can get a better result than fastai’s default.
  2. Try switching between AdaptiveConcatPool2d and AdaptiveAvgPool2d in a CNN head and see what difference it makes.
  3. Write your own custom splitter to create a separate parameter group for every ResNet block, and a separate group for the stem. Try training with it, and see if it improves the pet recognizer.
  4. Read the online chapter about generative image models, and create your own colorizer, super-resolution model, or style transfer model.
  5. Create a custom head using nearest neighbor interpolation and use it to do segmentation on CamVid.

In [ ]: