Coarse-to-Fine (CTF), hierarchical strategy, and cross-scale cost aggregation have efficiently expanded cost aggregation methods and yield a highly accurate disparity map to some extent. They are committed to providing a good trick to find the correct matching points in the weak texture region. However, these methods must be multi-scale as the prerequisite and usually need the assistance of image pyramid. They are limited to the propagation of errors from coarse to fine levels and poor recovery of thin structures. In this study, a generic fusion of gray scale cost aggregation framework is proposed which encourages the initial cost aggregation to integrate the cost aggregation of gray image. The main purpose of the gray image after Gaussian filter is to match the corresponding pixels in the weak texture region of the image better. Meanwhile, it does not need to scale down to build the image pyramid and aggregate cost at each scale and thus accelerate the step of cost aggregation. Furthermore, guided image filtering and fast weighted median filtering are introduced in this study for cost aggregation and disparity refine. In addition, to avoid choosing ambiguity that WTA (winner-take-all) brings, the interrelationship between the minimum value of cost aggregation and the second smallest value is utilized to determine the final disparity. It is shown that the fusion of gray scale cost aggregation framework is important as it effectively leads to significant improvement evaluated on Middlebury.