Abstract: Deep Reinforcement Learning (DRL) makes a big improvement for robotic control. However, training stable and efficient bipedal gaits like walking and running still remains challenging.