A face feature representation method based on difference local directional pattern (DLDP) is proposed. Firstly, each pixel of every facial image sub-block gains eight edge response values by convolving the local neighborhood with eight Kirsch masks. Then, the difference of each pair of neighboring edge response values is calculated to form eight new difference directions. The top k difference response values are selected and the corresponding directional bits are set to 1. The remaining (8-k) bits are set to 0, thus forming the binary expression of a difference local direction pattern. In addition, high-resolution Kirsch masks only consider directions but ignore the weight values of each pixel location. DLDP proposes a design method of weight values. Finally, the sub-histogram is calculated by accumulating the number of different DLDP of image blocks. All sub-histograms of an image are concatenated into a new face descriptor. Experimental results show that DLDP achieves state-of-the-art performance for difficult problems such as expression, illumination and occlusion-robust face recognition in most cases. Especially, DLDP gets better results for occlusion problem.