Differential evolution algorithm usually solves the constrained optimization problems by the feasible solutions priority rule, but the method can not use the infeasible solutions information populations. ε-DE algorithm is designed and can use the information of infeasible solutions. By designing new comparison rules, the infeasible solutions with better objective function are made full use of in the evolution process. The concept of population constraint relax degree is introduced in the comparison rules. During the evolution initial phase, the infeasible solutions with better objective function and near the boundary of the feasible region are incorporated in the population. With the evolutionary generation increasing, the decrease in the population constraint relax degree decreases the number of infeasible solutions in the population. Unless the population constraint relax degree is 0, the population is entirely composed of feasible solutions. In addition, an improved DE algorithm is chosen as the search algorithm, so a faster convergence is gotten. The simulation results of 13 benchmark functions prove that ε-DE is most competitive in all DE algorithms for solving COPs.