FSS(feature subset selection) is an important problem in the fields of machine learning and pattern recognition. Minimum FSS problem has been proved NP hard. However, existing heuristic algorithms are based on the consistency of positive and negative examples set, and a more optimal feature subset is hard to be produced under the noisy data in application to real-world domains. In this paper, from the degree of statistics, the effects of noisy data on FSS is analyzed firstly, and a concept of consistent feature subset which contains error rate is given. Then a heuristic algorithm——EFS (entropy based feature subset selection) based on information-theoretic entropy measure and Laplace error rate is presented. It is also applied to two real-world domains and is compared with GFS (greedy feature subset selection). The experimental results show that EFS can produce more representative feature subset, and can solve the noisy problem in the practical application effectively.