I recently read an intriguing article on the application of Deep Q-Learning in robotics titled "Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning" by Andy Zeng. In this paper, the author employs a Fully Convolutional Network (FCN) to predict pixel-wise values for every pixel in the heightmap, where each value represents the expected reward (Q-value) for the robot performing a pushing or grasping action at that specific pixel. I am particularly curious about how the Q-values are calculated, especially in the context of the Conv2D layer. Upon reviewing the code, I noticed that the network architecture used for computing Q-values is as follows: self.pushnet = nn.Sequential(OrderedDict([ ('push-norm0', nn.BatchNorm2d(2048)), ('push-relu0', nn.ReLU(inplace=True)), ('push-conv0', nn.Conv2d(2048, 64, kernel_size=1, stride=1, bias=False)), ('push-norm1', nn.BatchNorm2d(64)), ('push-relu1', nn.ReLU(inplace=True)), ('push-conv1', nn.Conv2d(64, 3, kernel_size=1, stride=1, bias=False)) ])) I’m very new to this field, and I would greatly appreciate it if you could provide an example (using a smaller input size for simplicity). Thank you! I have tried searching for information on Conv2D, but it seems to be primarily used for object detection and recognition. When I looked for its applications in Reinforcement Learning, the explanations were unclear and lacked detail." Continue reading...
