Stroke fragmentation is the core of the pen-based interaction. This paper presents a novel method of stroke fragmentation, which combines geometric features and HMM (hidden Markov model). Four geometric features are employed to describe the local geometry of strokes, and a HMM structure is designed to model the drawing context to describe the global. Furthermore, stroke data is compressed as much as possible with the least loss of information by means of global searching and the best matching algorithm. It can locate the segment point and judge the primitive type simultaneously with acceptable computation efficiency. Experimental results show the effectiveness of the proposed method.