In this paper, the authors present a new principle for designing a sort of multi-layer weight-sum-and -threshold neural networks. The design process consists of two steps. First, each class of the given training samples is covered by using neighbor coverings as less as possible. Then a neural network is specifically designed by an approach called alternative covering algorithm. The simulation results of two representative hard classification problems are shown. One is so called “two spirals separation” problem, the other is “the learning problem of infinitetraining samples”. These simulation results are given to illustrate the effectiveness of the new method.