Studies have shown that the adversarial example attack is that small perturbations are added on the input to make deep neural network (DNN) misbehave. Meanwhile, these attacks also exist in Chinese text sentiment orientation classification based on DNN and a method "WordHandling" is proposed to generate this kind of adversarial examples. This method designs a new algorithm aiming at calculating important words. Then the words are replaced with homonym to generate adversarial examples, which are used to conduct an adversarial example attack in black-box scenario. This study also verifies the effectiveness of the proposed method with real data set, i.e. Jingdong shopping and Ctrip hotel review, on long short-term memory network (LSTM) and convolutional neural network (CNN). The experimental results show that the adversarial examples in this study can mislead Chinese text orientation detection system well.