This paper investigates the decomposition of diagnosis problem and gives a theorem for decomposition and combination of the diagnosis. On the basis of the above work, an algorithm using gradual approach to decomposing the diagnosis problem is proposed. Besides, the correctness, completeness and complexity of the algorithm are proved in this paper. The experimental results indicate that the algorithm can apparently improve the effectiveness of diagnosing multi-output system. Comparing with the method of decomposition by assuming instantiations of some variables, the algorithm is more efficient and applies to more general diagnosis problems.