Two improved Elman neural networks, output-input feedback Elman network and output-hidden feedback Elman network are presented based on the Elman neural network. By using the output-input feedback Elman network as a passageway of the error back propagation, a recurrent back propagation control neural network model is developed. The stability of the improved Elman neural networks is proved in the sense of Lyapunov stability theory. The optimal adaptive learning rates are obtained, which can guarantee the stable convergence of the improved Elman networks. The ultrasonic motor is simulated by using the Elman and improved Elman networks respectively. Besides simulating the speed of the ultrasonic motor successfully, some useful results are also obtained. According to the results, the different network models based on the sampling situation in the fieldwork can be chosen. Numerical results show that the recurrent back propagation control neural network controller has good effectiveness for various kinds of reference speeds of the ultrasonic motor.