[Python] how do i demonstrate overfit on neural network? [closed]

I was preparing a small presentation on Neural Networks and I can't get the classic curve when you increase the complexity of the model and validation score first grows then drops down. Here is the NN I used:

class ConvolutionEMNISTModel(nn.Module):
def __init__(self, input_shape: int, hidden_units: int, output_shape: int):
super().__init__()
self.layer_stack = nn.Sequential(
nn.Conv2d(in_channels=input_shape,
out_channels=hidden_units,
kernel_size=3,
stride=1,
padding=1),
nn.ReLU(),
nn.Conv2d(in_channels=hidden_units,
out_channels=hidden_units,
kernel_size=3,
stride=1,
padding=1),
nn.ReLU(),
nn.MaxPool2d(kernel_size=2),
nn.Flatten(),
nn.LazyLinear(out_features=output_shape)
)

def forward(self, x):
return self.layer_stack(x)


As a model's complexity I used hidden_units varying from 4 to 4096 and tried few different datasets: MNIST, Fashion-MNIST, CIFAR-10. But whatever I do, I can't ever get validation score decreasing. In despair I even tried to set train set size to 100 and trained for 200 iterations - nothing helps. I'll be really grateful if anyone hints me what am I doing wrong.

Continue reading...