Let PyTorch find the gradients

Step 1

Autograd records the calculation

Autograd is PyTorch's automatic differentiation system. Mark a parameter with requires_grad=True, then calculate a loss using it. PyTorch records how those operations depend on the parameter. Calling loss.backward() works backward through that calculation and stores derivatives in .grad.

This is backpropagation: applying the chain rule backward through a computation graph. It calculates gradients; it does not update parameters by itself.

import torch

weight = torch.tensor(0.0, requires_grad=True)
bias = torch.tensor(1.0, requires_grad=True)
prediction = weight * 3.0 + bias
loss = (prediction - 7.0) ** 2
loss.backward()

print("Weight gradient:", weight.grad.item())
print("Bias gradient:", bias.grad.item())

Step 2

Predict → measure → backpropagate → update

Now train on the same five input/target pairs. We explicitly start new parameters at zero, so this cell does not reuse the previous example's gradients.

PyTorch accumulates gradients by default. Clear them before each fresh backward pass. Use torch.no_grad() during the manual update so changing a parameter is not recorded as part of the next prediction graph. .item() turns a one-element tensor into a Python number for printing.

Every iteration builds a fresh forward calculation. You normally do not call backward twice on the same loss object; calculate a new loss on the next iteration.

X = torch.tensor([[-2.0], [-1.0], [0.0], [1.0], [2.0]])
y = 2 * X + 1
weight = torch.tensor(0.0, requires_grad=True)
bias = torch.tensor(0.0, requires_grad=True)
learning_rate = 0.1

for epoch in range(60):
    weight.grad = None
    bias.grad = None
    predictions = weight * X + bias
    assert predictions.shape == y.shape
    loss = ((predictions - y) ** 2).mean()
    loss.backward()
    with torch.no_grad():
        weight -= learning_rate * weight.grad
        bias -= learning_rate * bias.grad
    if epoch % 20 == 0:
        print(f"Epoch {epoch:2d} | loss before update: {loss.item():.6f}")

print(f"Learned weight: {weight.item():.3f}, bias: {bias.item():.3f}")

Step 3

Use the model without training it

Inference means using learned parameters to make predictions. We do not need to record gradients while doing that. torch.no_grad() reduces unnecessary tracking; it does not erase the learned numbers.

In the next lesson, an optimizer will handle the update and a neural-network module will hold the parameters. Keep this manual version nearby as a map of what those tools do.

with torch.no_grad():
    new_inputs = torch.tensor([[3.0], [4.0]])
    new_predictions = weight * new_inputs + bias
print("New predictions:", new_predictions.flatten().tolist())