Abstract:Previous multi-label learning requires that all or at least a subset of ground truth labels is given for the training example. This study investigates how to utilize the wisdom of crowds for multi-label tasks, where rather than high cost ground truth labels, imperfect annotations from crowds are collected for learning. The target is to infer the instances’ ground truth labels. The key challenge lies in how to aggregate the noisy annotations. Different from previous crowdsourcing works on single-label problems which ignore the correlation between labels, and multi-label works which consider local label correlations whose estimation heavily depends on the annotations’ quality and quantity, this study proposes an approach considering the global low rank structure of the crowds’ annotations. Regarding the crowds’ annotations for multi-label tasks as a three-way tensor (instance, label, worker), the crowds’ annotations are firstly preconditioned using low rank tensor completion, such that it is able to simultaneously correct the observed noisy annotations and at the same time predict the unobserved annotations. After that, the preconditioned annotations are aggregated through some heuristic methods. Three aggregation methods taking into account the crowds’ annotation confidence from different perspectives are tested. Experimental results on real world multi-label crowdsourcing data sets demonstrate the superiority of the proposed approach.