Learning from examples is to obtain a general rule through induction from a given set of positive and negative examples of a concept, which may describe all the positive examples, and reject all the negative examples of that concept. According to the extension matrix theory, to discover the equations which satisfy all positive examples on the background of negative examples may be considered as to find a path within the matrix of negative examples. In order to deal with the multi-class problems with overlay, an improved extension matrix algorithm has been proposed in this paper. The heuristic search based on average entropy has been used to get the approximate solutions of the shortest equation. The potential function is used to estimate the probability density function of the overlay area between positive and negative examples, so that the non-linear interfaces of the interclass areas may be obtained. The improved algorithms have been applied to handwritten Chinese character recognition and its effectiveness has been proved through comparison study and analysis.