[Python] Unable to reproduce training results in a dummy vector using stablebaseline3

I created a custom Gymnasium environment and trained an agent using Stable-Baselines3 with DummyVecEnv and VecNormalize. The agent performs well during training and consistently reaches the goal. However, in testing, it fails to replicate the same behavior.

I'm using TD3 for training:

model = PPO("MlpPolicy", env, ...)
model.learn(...)


And PPO for testing:

env = StewartGoughEnv()
env = DummyVecEnv([lambda: env])
env = VecNormalize.load(env_path, env)
model = PPO.load(model_path, env=env)


Despite training success, the agent doesn’t perform well in evaluation. Any idea why this might be happening?

I tried chatgpt suggestions to save the environment also but that did not help at all! I can observing the problem solved in the training stage but not in the testing stage!

Continue reading...