With the development of deep learning technologies such as Transformer-based pre-trained models, breakthroughs have been made in the research and applications of large language models (LLMs) have shown great understanding ability and creativity, which not only have an important impact on downstream tasks such as abstractive summarization, dialogue generation, machine translation, and data-to-text generation, but also show broad application prospects in multimodal fields such as image annotation and visual narrative. While LLMs have significant advantages in performance, deep learning-based LLMs are prone to hallucinatory problems, which would reduce the system performance, and even seriously affect the faithfulness and broad applications of LLMs. The accompanying legal and ethical risks have become the main obstacles to their further development and implementation. Therefore, this survey provides an extensive investigation and technical review of the hallucination problem in LLMs. Firstly, the hallucination problems in LLMs are systematically summarized, and their origin and causes are analyzed. Secondly, a systematical overview of hallucination evaluation and mitigation methods is provided, in which the evaluation and mitigation methods are categorized and thoroughly compared for different tasks. Finally, the future challenges and research directions of LLMs’ hallucination are discussed from the evaluation and mitigation perspectives.