This paper proposes an effective method for computing optimal state transition probabilities for software reliability estimation based on a Markov usage model. This method uses Cross-Entropy to measure the differences between the operational profile and the sampling distribution with zero variance. By adjusting the probabilities of state transitions during test, an iterative method based on the Cross-Entropy is proposed for this choice, and an unbiased reliability estimator with zero variance is obtained. Simulation results show that the testing profile with Cross-Entropy method performs significantly better than the simulated annealing algorithm. Moreover, this strategy can more effectively accelerate software statistical testing.