In this paper, the authors present a novel method to incorporate temporal correlation into a speech recognition system based on conventional hidden Markov model (HMM). The temporal correlation is considered to be useful for recognition because of the fact that the speech features of the present frame are highly informative about the feature characteristics of neighboring frames. An obvious way to incorporate temporal correlation is to condition the probability of the current observation on the current state as well as on the previous observation and the previous state. But using this method directly must lead to unreliable parameter estimation for the number of parameters to be estimated may increase too excessively to limited train data. In this paper, the authors approximate the joint conditional PD by non-linear estimation method. As a result, they can still use mixture Gaussian density to represent the joint conditional PD for the principle of any PD can be approximated by mixture Gaussian density. The HMM incorporated temporal correlation by non-linear estimation method, which they called FC (frame correlation) HMM does not need any additional parameters and it only brings a little additional computing quantity. The results of the experiment show that the top 1 recognition rate of FC HMM has been raised by 6 percent compared to the conventional HMM method.