Pose and illumination changings from picture to picture are two main barriers toward full automatic face recognition. In this paper, a novel method to handle both pose and lighting conditions simultaneously is proposed, which calibrates the pose and lighting to a predefined reference condition through an illumination invariant 3D face reconstruction. First, some located facial landmarks and a priori statistical deformable 3D model are used to recover an elaborate 3D shape. Based on the recovered 3D shape, the “texture image” calibrated to a standard illumination is generated by spherical harmonics ratio image and finally the illumination independent 3D face is reconstructed completely. The proposed method combines the strength of statistical deformable model to describe the shape information and the compact representations of the illumination in spherical frequency space, and handles both the pose and illumination variation simultaneously. This algorithm can be used to synthesize virtual views of a given face image and enhance the performance of face recognition. Experimental results on CMU PIE database show that this method can significantly improve the accuracy of the existing face recognition method when pose and illumination are inconsistent between gallery and probe sets.