Recently, with the rapid development of imaging and genomic techniques, the brain imaging genetics has received extensive attention. In the brain imaging genetic studies, it is a challenging task to examine the influence of genetic variants, i.e., single nucleotide polymorphisms (SNPs), on structures or functions of human brains. In addition, multimodal brain imaging phenotypes extracted from different perspectives and imaging markers from the same region consistently showing up in multimodalities gives more ways to understand the diseases mechanism, such as Alzheimer's disease (AD). Accordingly, This work exploits multi-modal brain imaging phenotypes as intermediate traits to bridge genetic risk factors and disease status. Consistent phenotype between genetic risk factors and disease status is discovered via the designed label-aligned multi-modality regression method in AD. Specifically, standard multi-modality method is first applied to explore the relationship between the well-known AD risk SNP APOEe4 rs429358 and multimodal brain imaging phenotypes. Secondly, to utilize the label information among labeled subjects, a new label-aligned regularization is included into the standard multi-modality method. In such way, all multimodality subjects with the same class labels should be closer in the new embedding space. Finally, the experiments are conducted on three baseline brain imaging modalities, i.e., voxel-based measures extracted from structural magnetic resonance imaging, fluorodeoxyglucose positron emission tomography and F-18 florbetapir PET scans amyloid imaging, from the Alzheimer's disease neuroimaging initiative (ADNI) database. Related experimental results validate that the proposed method can identify robust and consistent regions of interests over multi-modality imaging data to guide the disease-induced interpretation. Furthermore, the values of correlation coefficient have been increased by 8%, 9%, and 5% in comparison with the best results of the existing algorithms on three modalities.