As mobile data is growing everyday, how to predicate the wireless traffic accurately is crucial for the efficient and sensible allocation of communication and network resources. However, most existing prediction methods use a centralized training architecture, which involves large-scale traffic data transmission, leading to security issues such as user privacy leakage. Federated learning can train a global model with local data storage, which protects users’ privacy and effectively reduces the burden of frequent data transmission. However, in wireless traffic prediction, the amount of data from the single base station is limited, and the traffic patterns vary among different base stations, making it difficult to capture the traffic patterns and resulting in poor generalization of the global model. In addition, traditional federated learning methods employ averaging in model aggregation, ignoring the differences in guest contributions, which further leads to the degradation of the global model performance. To address the above issues, this study proposes an attention-based “intra-cluster average, inter-cluster attention” federated wireless traffic prediction model. The model first clusters base stations based on their traffic data to better capture the traffic variation characteristics of base stations with similar traffic patterns. At the same time, a warm-up model is designed to alleviate data heterogeneity by a small amount of base station data to improve the generalization ability of the global model. The study introduces the attention mechanism in the aggregation stage to quantify the contributions of different objects to the global model and incorporates the warm-up model in the model iteration process to improve the prediction accuracy of the model. Extensive experiments are conducted on two real-world datasets (Milano and Trento), and the results show that the DualICA outperforms all baseline methods. The mean absolute error performance gain over the state-of-the-art method is up to 10.1% and 9.6% on the two datasets, respectively.