[Python] Cannot understand the cause for a sudden spike in calls in my neural network

I've been working on my first neural network project. It's a two-people, 3-card game poker, here between my network and a logical rule-based bot that never bluffs. I am using a numpy implemented vanilla REINFORCE policy with a moving-average baseline and gradient-ascent on log-prob. It's a one hidden, 70 neurons, layer with ReLU, with softmax at the end for the action probabilities.

Here are two graphs: Graph 1 with learning rate 5e-6 and Graph 2 with lr 1e-5 containing spikes in call rates. They both have the random seeds are different.

While the bluff rate and value bet rates stay constant, the call rate always suddenly changes from 0.3 to 0.5, which in turn reduces the This is a recurrent issue I face, and I cannot explain why.

I would be very grateful for any insight or hypotheses that would explain the causes. Thank you

Continue reading...