Activity recognition has many potential applications in pervasive computing field, such as medical care, intelligent home/office, business service, etc. Recently, sensor based activity recognition has attracted much attention, due to its advantages of ubiquity and less intrusion. This paper studies sensor based activity recognition. In particular, it recognizes high-level user activities from multi-mode sensor data using machine learning methods. This paper proposes an activity recognition framework, which makes use of low-level wireless network signal and acceleration data to answer questions like “where is the user”, “what is the user doing” and “what is the user going to do”. Three main fusion algorithms are included in this framework. Considering the trend of pervasive computing, this paper uses the mobile phones and their embedded sensors to collect activity information. Effectiveness of the proposed algorithms is confirmed on real world collected dataset.