Handheld mobile computing has increasingly become a mainstream computing paradigm, in which human-computer interaction is an essential issue. Multi-Modal user interface and multi-modal interaction can effectively improve the interaction efficiency in mobile computing. This paper presents a multi-modal interaction model and discusses the application of multi-modal interaction in handheld devices. Then the tour guide system is described. An experimental evaluation shows that the multi-modal interaction, based on pen and speech, can greatly improve its interaction.