Lab 4 - Math 178, Spring 2024#

You are encouraged to work in groups of up to 3 total students, but each student should make a submission on Canvas. (It’s fine for everyone in the group to submit the same link.)

Put the full names of everyone in your group (even if you’re working alone) here. This makes grading easier.

Names:

Train an XOR network using PyTorch#

  • Use PyTorch to train a neural network which produces a perfect (4 out of 4) prediction rate for XOR. (This is similar to what you did “by hand” on question 2d on Homework 3. You should not be manually setting the weights, but instead, should be using PyTorch to find weights. Use a Binary Cross Entropy loss function. Feel free to use a more complex Neural Network architecture than what you did by hand in the homework. I was able to eventually get a small neural network architecture to work, but I had to re-run the code numerous times.)

Recommended references:

  1. I primarily used the attached University of Washington notebook, which I downloaded from Google Colab.

  2. I personally solved this exercise before reading through the PyTorch tutorial which will be used below. If I had started with that tutorial, maybe I would have used a different approach. But that tutorial is fancier than what we need here, because of the data loaders etc.

Comment:

  1. The MNIST portion below is probably easier, in terms of what you need to do, but I’m putting this part first because the resulting neural network here is conceptually simpler.

Train an MNIST network using PyTorch#

  • Adapt the code at the PyTorch tutorial to train an MNIST neural network. Adjust parameters as necessary to reach at least a 91% test accuracy. (Be sure you’re using datasets.MNIST rather than what’s in the tutorial: datasets.FashionMNIST. Most other parts of the tutorial should adapt easily. I deleted the the GPU parts such as if torch.cuda.is_available() because I don’t think they will work on Deepnote, but perhaps they are useful also here.)

Submission#

  • Using the Share button at the top right, enable public sharing, and enable Comment privileges. Then submit the created link on Canvas.

Possible extensions#

  • My code for the small XOR neural network only works about one out of ten times. Can you produce code that always works, for example, using a while loop or some change to the hyperparameters (not including making the hidden layer bigger)?

  • Neural networks are very prone to overfitting. Does that happen with your MNIST code? Can you plot a train and test error curve to demonstrate? How can you combat overfitting, for example, using nn.Dropout?

  • Can you use Python (don’t try to do this by hand) the decision boundary for the XOR classifier from part 1? Here is some sample code from Math 10 for drawing decision boundaries: https://christopherdavisuci.github.io/UCI-Math-10-S23/Week8/Week8-Wednesday.html

Created in deepnote.com Created in Deepnote