Abstract:The coarse-grained scheduling used in cloud computing platform allocates fixed quantity resources to tasks. However, this allocation can easily lead to problems such as resource fragmentation, over-commitment and inefficient resource utilization. This study proposes a dynamically fine-grained scheduling method to resolve those problems. This method estimates resource requirement of task according to similar tasks and divides tasks into execution stages according to the task requirement, and it also matches task resource requirement and available server resources by stages to refine two aspects of allocation granularity: allocation duration and allocation quantity. Furthermore, this method may compress resource allocation to further improve resource utilization and performance, and this method uses several mechanisms including runtime resource monitoring, allocation policy adjustments, and scheduling constraint checks to ensure resource utilization and performance of cloud computing platform. Based on this method, a scheduler has been implemented in the open source cloud computing platform Yarn. The test results show that the dynamically fine-grained scheduling method can resolve resource allocation problems by significantly improving resource utilization and performance with acceptable fairness and scheduling response times.