[Python] Using Pytorch Sequential for reinforcement model without gym

I'm trying to play a game with AI, but I want to do it in real time. Because of that, I'm not using gym to create an environment.

I want to take a screenshot, preprocess it, then pass it through the model before updating the model based on whether or not the model died in-game.

from PIL import ImageGrab, Image
import numpy as np
import torch
from torch import nn
from torchvision import transforms
import torch.optim as optim
# imporot keyboard and pyautogui for playing the game

_bbox=(800, 200, 1200, 900) # (L,T,R,B)


def get():
screenshot = ImageGrab.grab(bbox=_bbox)

# Check if dead
if np.sum(np.array(screenshot)[:,:,0])/(screenshot.size[0]*screenshot.size[1])>=190 and np.sum(np.array(screenshot)[:,:,1])/(screenshot.size[0]*screenshot.size[1])<50 and np.sum(np.array(screenshot)[:,:,2])/(screenshot.size[0]*screenshot.size[1])<15:
print("Died") # Do something else than printing "Died"

return screenshot.copy()


# define preprocessor
preprocess = transforms.Compose([
transforms.Resize(256),
transforms.CenterCrop(256),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])

# define model (am I doing this right?)
model = nn.Sequential(
nn.Conv2d(3,256,5),
nn.ReLU(),
nn.MaxPool2d(2, 2),
nn.Conv2d(256,512,5),
nn.MaxPool2d(2, 2),
nn.ReLU(),
nn.Flatten(),
nn.Linear(3721, 512),
nn.LeakyReLU(),
nn.Linear(512, 512),
nn.LeakyReLU(),
nn.Linear(512, 3),
nn.Softmax(1)
)

# what I want to do
inputs = preprocess(get(raw=True))
outputs = model(inputs)
pred = torch.argmax(outputs.data[0]).item() # 0 = no moves, 1 = left, 2 = right
# do move in game (I know how to do this part)
# reward/punish model if it died (I can use time alive or a progress score, but only after it failed)


I need assistance for that last part, but please feel free to give your input on the rest of the code as well. I don't really know what I'm doing.

Continue reading...