← Deep learning course

Deep learning / Lesson 06 of 06 / 45–60 min

Test it, save it, make it yours

Finish a small classifier with separate training, validation, and test examples, then save and reload its learned weights.

By the end

Complete a reproducible mini-project and distinguish fitting training data from doing well on unseen examples.

Step 1

Give the network examples it has not seen

Our final project classifies two-dimensional points: class 1 when their coordinates have opposite signs, otherwise class 0. This is a continuous cousin of XOR. The rule generates our teaching labels; the network does not receive the rule as an input.

We create 400 synthetic examples, then use a fixed random permutation to split them into training (240), validation (80), and test (80). Training examples drive parameter updates. Validation examples help choose settings such as hidden size or epoch count. Test examples are reserved for the final evaluation.

These are independent synthetic points from one simple distribution. Success here does not establish performance on photographs, language, or real-world decisions. For real datasets, also watch for duplicate examples and related people, time periods, or sources leaking across splits.

PyTorch · run in your notebook
import torch
from torch import nn

torch.manual_seed(7)
X = torch.rand(400, 2) * 2 - 1
y = (X[:, 0] * X[:, 1] < 0).float().reshape(-1, 1)
order = torch.randperm(len(X))
train_ids, val_ids, test_ids = order[:240], order[240:320], order[320:]
X_train, y_train = X[train_ids], y[train_ids]
X_val, y_val = X[val_ids], y[val_ids]
X_test, y_test = X[test_ids], y[test_ids]
print("Split sizes:", len(X_train), len(X_val), len(X_test))

Step 2

Use validation to understand learning

We use the same model shape as the XOR lesson, with 16 hidden neurons for this larger set. Set the main experimental choices near the top: hidden size, learning rate, and epochs.

The helper function measures loss and accuracy without updating parameters. Accuracy counts how many thresholded predictions match the targets; loss also reflects the scores behind those decisions.

Overfitting happens when fitting the training examples does not translate to unseen data. Falling training loss alongside rising validation loss is a warning sign. This clean toy dataset may not show strong overfitting, and we should not pretend it does. Compare the actual measurements you get.

PyTorch · run in your notebook
hidden_size = 16
learning_rate = 0.03
epochs = 400
model = nn.Sequential(nn.Linear(2, hidden_size), nn.Tanh(), nn.Linear(hidden_size, 1))
loss_fn = nn.BCEWithLogitsLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)

def evaluate(inputs, targets):
    model.eval()
    with torch.no_grad():
        logits = model(inputs)
        loss = loss_fn(logits, targets).item()
        predicted = (torch.sigmoid(logits) >= 0.5).float()
        accuracy = (predicted == targets).float().mean().item()
    return loss, accuracy

history = []
for epoch in range(epochs):
    model.train()
    optimizer.zero_grad()
    loss = loss_fn(model(X_train), y_train)
    loss.backward()
    optimizer.step()
    if (epoch + 1) % 100 == 0:
        train_loss, train_accuracy = evaluate(X_train, y_train)
        val_loss, val_accuracy = evaluate(X_val, y_val)
        history.append((epoch + 1, train_loss, val_loss))
        print(f"Epoch {epoch + 1} | train loss {train_loss:.3f} | "
              f"val loss {val_loss:.3f} | val accuracy {val_accuracy:.1%}")

Step 3

Do a final test and preserve what you learned

Choose your settings using validation first. Then run this section once for a final test report. If you repeatedly tune based on the test result, it is no longer an untouched test.

We save a state dictionary containing learned parameters, plus the hidden size needed to recreate the architecture. A new model loads those parameters and should make the same predictions. This is an inference checkpoint, not a full training-resume checkpoint: resuming Adam training would also require its optimizer state.

The file is written as room310_tiny_net.pt in your runtime's working directory. In Colab, download it from the Files sidebar before the runtime resets. Only load model files you trust; our example reloads the file it just created.

PyTorch · run in your notebook
test_loss, test_accuracy = evaluate(X_test, y_test)
print(f"Final test | loss {test_loss:.3f} | accuracy {test_accuracy:.1%}")

torch.save({"hidden_size": hidden_size, "model_state": model.state_dict()},
           "room310_tiny_net.pt")
checkpoint = torch.load("room310_tiny_net.pt", map_location="cpu", weights_only=True)
restored = nn.Sequential(
    nn.Linear(2, checkpoint["hidden_size"]), nn.Tanh(),
    nn.Linear(checkpoint["hidden_size"], 1),
)
restored.load_state_dict(checkpoint["model_state"])
restored.eval()
with torch.no_grad():
    assert torch.allclose(model(X_test), restored(X_test))
    new_points = torch.tensor([[-0.8, 0.7], [0.7, 0.6]])
    probabilities = torch.sigmoid(restored(new_points))
print("Reload check passed. New p(class 1):", probabilities.flatten().tolist())

Your turn

Practice & explain

Use the practice section at the end of this lesson's Colab notebook, or work in your local PyTorch environment. Keep a copy of your changes.

6.1 · Your mini-project report

Compare hidden sizes 4 and 16 using validation only. For each run, start from the top and record the seed, learning rate, epoch count, final train loss, validation loss, and validation accuracy. Choose one configuration, report its final test accuracy, and save its checkpoint.

Need a hint?

Keep all settings except hidden size the same. If you already used the test set for tuning, say so and generate a fresh independent final test set.

6.2 · Explain what you built

Write five sentences: what the inputs and labels represent; what the layers do; how loss and gradients change parameters; how you prevented test leakage; and one limitation of the model. Include two new-point predictions and the saved file.

Need a hint?

Passing a reload check proves the weights were restored, not that the classifier is accurate. Keep those two claims separate.

Open in Colab ↗Download notebookDownload Python file

Keep exploring