Software statistical testing is concerned with testing software systems based on their usage models. In the context of UML(unified modeling language)-based development, it is desired that the usage models can be derived from the UML analysis artifacts. This paper presents a method that derives the software Markov chain usage models from the reasonably annotated UML artifacts. The method utilizes the annotated use case diagrams, the annotated sequence diagrams, and the use case execution sequence relations. These annotations and the use case execution sequence relations are called statistical testing constraints. The method presents an algorithm that derives the Markov chain usage models from the UML artifacts and the statistical testing constraints. The framework of the support tool, UMGen, is also presented. The usage models can be used to generate software statistical test cases and estimate the software reliability.