The paper studies a matrix factorization problem for time series data, where the target matrix R consists of the equal length time series data generated by a set of objects. The goal is to find two low rank matrices U and V, such that R≈U^T×V. Many time series analysis problems, such as finance data analysis and missing traffic data imputation, can be reduced to the proposed model. A probabilistic graphical representation for the problem is proposed, and a constrained optimization model from the graphical representation is derived. The solution algorithms for the proposed model is also presented. Empirical studies show that the proposed model is superior to the baselines.