Key-Value stores play an important role in today’s large-scale, high-performance cloud applications. Elastic scaling and load rebalancing with low cost live data migration are critical to enabling the elasticity of Key/Value stores in the cloud. Learning how to reduce the migration cost is one difficult problem that cloud providers are trying to solve. Many existing works try to address this issue in non-virtual environments. These approaches, however, may not be well suited for cloud-based Key/Value stores. To address this challenge, the study tackles the data migration problem under a load rebalancing scenario. The paper builds a one cost model that could be aware of the underlying VM interference and trade-off between migration time and performance impact. A cost-aware migration algorithm is designed to utilize the cost model and balance rate to guide the choice of possible migration actions. Our evaluation using Yahoo! Cloud Serving Benchmark shows the effectiveness of the approach.