Open-set recognition is an important issue for ensuring the efficient and robust deployment of machine learning models in the open world. It aims to address the challenge of encountering samples from unseen classes that emerge during testing, i.e., to accurately classify the seen classes while identifying and rejecting the unseen ones. Current open-set recognition studies assume that the covariate distribution of the seen classes remains constant during both training and testing. However, in practical scenarios, the covariate distribution is constantly shifting, which can cause previous methods to fail, and their performance may even be worse than the baseline method. Therefore, it is urgent to study novel open-set recognition methods that can adapt to the constantly changing covariate distribution so that they can robustly classify seen categories and identify unseen categories during testing. This novel problem adaptation in the open world (AOW) is named and a test-time adaptation method is proposed for open-set recognition called open-set test-time adaptation (OTA). OTA method only utilizes unlabeled test data to update the model with adaptive entropy loss and open-set entropy loss, maintaining the model’s ability to discriminate seen classes while further enhancing its ability to recognize unseen classes. Comprehensive experiments are conducted on multiple benchmark datasets with different covariate shift levels. The results show that the proposal is robust to covariate shift and demonstrates superior performance compared to many state-of-the-art methods.