I've recently been trying to make a Pong agent using reinforcement learning, but I keep getting errors no matter what I do. The PPO implementation I’m using was originally designed for CartPole, and I modified it to work with Pong. The problem likely comes from this part of model: with torch.no_grad(): dummy = torch.zeros(1, 1, 96, 96) # batch=1, grayscale, 96x96 conv_out = self.shared_layers(dummy) flattened_size = conv_out.view(1, -1).shape[1] # Inside ActorCriticNetwork.__init__() with torch.no_grad(): dummy = torch.zeros(1, 1, 96, 96) # simulate one preprocessed image conv_out = self.shared_layers(dummy) flattened_size = conv_out.view(1, -1).shape[1] and how it connects to the Linear layer: self.policy_layers = nn.Sequential( nn.Linear(flattened_size, 64), # ← uses flattened_size nn.ReLU(), nn.Linear(64, action_space_size) ) Full code: https://github.com/twhighschooler/pong Continue reading...
