Personal group information on social media is useful for understanding social structures. Existing studies mainly focus on detecting personal groups using explicit social information between users, but few pay attention on using implicit social information and textual information. In this paper, a latent factor graph model (LFGM) is proposed to recommend personal groups for each person with both explicit and implicit information from textual content and social context. Especially, while explicit textual and social contents can be easily extracted from user generated content and personal friendship information, a matrix factorization approach is applied to generate both implicit textual and social information. Evaluation on a large-scale dataset validates the effectiveness of the proposed approach.