The generative pertained transformer-based large language models (LLMs) are setting off a wave in the field of artificial intelligence and continue to penetrate their influence into more fields. The LLMs such as ChatGPT have demonstrated their ability and potential to provide people with a certain degree of assistance through natural language-based interaction in many software engineering tasks, and they are developing into a natural language-based human-machine collaborative tool for software development and evolution. From the perspective of human-machine collaborative software development and evolution, the LLMs, as a software tool, present two major features. One is the natural language-based human-machine interaction, which greatly expands the human-machine collaboration workspace and improves the efficiency and flexibility of human-machine collaboration. The second is to generate predictive contents based on accumulated knowledge of software development and evolution, targeting a given software development and evolution task, which can provide a certain degree of support and assistance for the software development and evolution task. However, since LLMs are essentially mathematical models based on probability and statistical principles and training date, with inexplicability and uncertainty, the contents generated by LLMs are predictive and lack the judgments for trustworthiness. As opposed to the tasks that humans need to perform in software development and evolution, which are typically decision-making tasks with trustworthiness guarantees, LLMs, as a software tool, not only provide assistance to people in software development and evolution featuring human-machine collaboration but also bring many challenges. This study analyzes and clarifies the challenges brought by the LLMs, such as how to construct LLMs that are more helpful for software development and evolution, how to guide LLMs to generate predictive contents that are more helpful for software development and evolution, and how to develop and evolve high-quality software systems based on the predictive contents generated by LLMs.