[Python] Best path finding/planning algorithm to use for RL agent problem in stochastic speed...

In my project I have an environment with an agent navigating on it, trying to get to the goal line on the right side. Some parts of the environment are faster than others (dark areas are labeled fast, white areas labeled slow). I have a heuristic approximation that is able to approximate location of the robot (not super precise but good enough) and approximate its speed. According to that I create a 2D state matrix which has high positive values (dark blue) for dark states where robot can accelerate very fast, small positive values (0-10, light blue) where the robot is going at good enough speed, and negative (red) value where the robot stops and gets stuck or is going too slowly (red areas).

I am using CEM to stochastically choose actions and movement vectors to explore the environment and Im able to get 65%-80% of the environment explored during my training which is good enough, in all the cases I tested so far I was able to visually see various possible path to the final goal line:
However Im wondering what algorithm to use when doing the testing once I know my state values. Visually Im always able to find a good quick path, prioritizing speed and time it takes to complete the path, however all the methods I tried whether its CME or some dynamical value iteration, all failed and at some point the agent would either end up getting stuck or going through a very slow section such as in this screenshot (green lines are various paths generated during training and orange is during testing, started of well but then instead of following positive values it went straight into unexplored or negative area)

As mentioned I tried CME and some forms of DP however are there any other algorithms you think would work well here? I want to make sure it never get stuck in a negative state, finds the optimal path prioritizing speed and preferably takes actions heading to the areas that have high proportion of the positive states (ie doesnt get too close to negative states in general) Thanks for any tips!

Continue reading...