Introduction

The goal of this lab is to use a pretrained convolutional neural network to perform a custom computer vision task.

Specifically, we will take a ResNet trained on ImageNet and use it to classify CIFAR10. The ImageNet model has 1000 classes whereas CIFAR10 has only 10, so we cannot use the model out of the box. We will replace the classification head and retrain only that head, leaving the rest of the network frozen.

Starter code is provided in resnet-lab.ipynb [html].

This is the last lab of the course, and it closes a loop. In Lab 4 you designed a feature by hand. In Lab 7 you designed a much better one, also by hand. Here you will use features that nobody designed — they were learned from data — and you will find that they outperform everything you built, on a task they were never trained for.

Reading

Computer Vision: Algorithms and Applications (2nd ed.), Szeliski — Sec. 5.1.3, 5.3, 5.4, 6.2.2.

Logistic regression and deep neural networks (the Week 11 material you will use for the baselines); convolutional neural networks; deep networks for image classification.

Data

CIFAR10 — 60,000 32x32 colour images in 10 classes. You do not need to download it by hand; torchvision.datasets.CIFAR10(root=..., download=True) fetches it (about 170 MB) the first time you run it and caches it thereafter.

You do not need a GPU for this lab. The backbone stays frozen, so you are only training a single linear layer. If training feels slow, reduce the number of epochs or the input resolution rather than the size of the dataset — and say what you changed.

Tasks

1. Get a pretrained model

Download a pretrained ResNet using timm (see https://timm.fast.ai/). Print the model architecture and identify the classifier head.

2. Swap the head

Replace the classifier head (the fc layer) with your own layer appropriate for 10 classes.

• Freeze all the other parameters.
• Report how many parameters are trainable versus frozen.

3. Train and evaluate

Train the new head on CIFAR10 and report test accuracy.

Note that ResNet expects a different input size and normalization than raw CIFAR10 images provide. Handle this properly and describe what you did.

4. Compare against a baseline you train yourself

A number on its own means nothing. Train two cheap baselines on the same data and the same train/test split:

1. Raw pixels. A 10-way linear classifier (logistic regression) on the flattened \(32 \times 32 \times 3\) images.
2. A hand-crafted feature. Compute an orientation histogram for each image — the same construction you used in Lab 4, and the same idea underlying the descriptor you built in Lab 7 — and train the same linear classifier on those instead.

Then fill in:

Then answer:

• Does the hand-crafted feature beat raw pixels? By how much?
• The pretrained ResNet never saw CIFAR10 during pretraining, and you retrained only the final layer. Why does it beat a feature that a human designed specifically for images?
• All three rows use the same linear classifier. So what exactly is the difference between them?

5. Look at the features

Extract the penultimate-layer activations (the input to the classifier head) for a few hundred CIFAR10 test images. Project them to 2D using PCA and plot them, coloured by true class.

Do the classes separate? Now do the same with the raw pixels of the same images. Show both plots side by side. This comparison is the single most important picture in this lab — discuss it.

6. Optional

Unfreeze the last residual block and fine-tune it along with the head. Does accuracy improve? What happens to training time?

Deliverables

Your notebook must contain the following.

• The pretrained model loaded and its head replaced, with trainable versus frozen parameter counts reported.
• A description of how you handled input resizing and normalisation.
• Test accuracy for the ResNet-with-new-head.
• The three-row comparison table — raw pixels, orientation histogram, and pretrained CNN features — with all three baselines actually trained on the same split.
• Answers to the three questions, including what differs between the rows given that the classifier is identical.
• The two PCA plots side by side: penultimate CNN activations, and raw pixels, both coloured by class.

Submission

Via Canvas. Please submit a single executed Jupyter notebook — one that has been run top to bottom, so that every figure and number listed above is visible in the submitted file. Code that has not been executed cannot be marked.

Parting thoughts

• Freezing the backbone is what makes this lab feasible without a GPU. If you unfreeze everything, you are training a ResNet from scratch, and you will be waiting a long time.
• “Pretrain on a large dataset, then adapt to your small one” is the dominant recipe in modern computer vision. You have just done it in about thirty lines of code.
• Where this goes next — vision transformers, foundation models, and generative image models — is the subject of Computer Vision II.