An improved genetic algorithm based on the evolutionarily stable strategy is proposed to avoid the problem of local optimum. The key to this algorithm lies in the construction of a new mutation operator controlled by a stable factor,, which maintains the polymorphism in the colony by setting a stable factor and changing certain best seeds to mutant. Therefore, the operator can keep the number of the best individuals at a stable level when it enlarges the search space. The simulation experiments show that this algorithm can effectively avoid the premature convergence problem caused by the high selective pressure. Moreover, this algorithm improves the ability of searching an optimum solution and increases the convergent speed. This algorithm has extensive application prospects in many practical optimization problems.