MAGMA is an open source high performance linear algebra package first developed for next-generation of heterogeneous/ hybrid architectures (CPUs+GPUs) with a dense linear algebra library similar to LAPACK in functionality, data storage, and interface. This paper presents performance testing and analysis of MAGMA. It first studies the matrix decomposition algorithm in MAGMA, then provides some useful suggestions of MAGMA usage and optimization through massive testing and source code analysis, and finally proposes a method for auto-tuning matrix decomposition block algorithms. In this test, the speedup is 1.09 for SGEQRF of square matrix and 1.8 for CGEQRF in terms of tall and skin matrix.