The maximum entropy approach is proved to be expressive and effective for the statistics language modeling, but it suffers from the computational expensiveness of the model building. An improved maximum entropy approach which makes use of mutual information of information theory to select features based on Z-test is proposed. The approach is applied to Chinese word sense disambiguation. The experiments show that it has higher efficiency and precision.