With the development of machine learning and application of big data, semantic-based emotional computing and analysis technology plays a significant role in the research on human perception, attention, memory, decision-making, and social communication. It affects not only the development in artificial intelligence technology, but also human/machine interaction and smart robot technology, therefore drawing widespread interest from the academic and business communities. In this paper, based on the definition of affection and the analysis of more than 90 emotional models, six vital problems and challenges in emotional computing are summarized as follows:where is emotion stem from and how to represent their essential features; how to analyze and compute the emotion under the multi-model environment; how to measure the influence of external factors on the process of emotional evolution; how to measure individual emotion by various of personalized characteristic; how to measure the crowed psychology and emotion and to analyze the mechanism about propagation dynamics; and how to express the subtle emotion and optimize algorithms. Meanwhile, some theoretical research, technical analysis and practical application are brought up to introduce the current work progress and trend for these technical challenges in order to provide new research clues and directions for further study in the field of the semantic-based emotional computing.