A meta-active learning approach exploiting instance importance

Active learning is focused on minimizing the effort required to obtain labeled data by iteratively choosing fresh data samples for training a machine learning model. One of the primary challenges in active learning involves the selection of the most informative instances for labeling by an annotation oracle at each iteration. A viable approach is to develop an active learning strategy that aligns with the performance of a meta-learning model. This strategy evaluates the quality of previously selected instances and subsequently trains a machine learning model to predict the quality of instances to be labeled in the current iteration. This paper introduces a novel approach to learning for active learning, wherein instances are chosen for labeling based on their potential to induce the most substantial change in the current classifier. We explore various strategies for assessing the significance of an instance, taking into account variations in the learning gradient of the classification model. Our approach can be applied to any classifier that can be trained using gradient descent optimization. Here, we present a formulation that leverages a deep neural network model, which has not been extensively explored in existing learning-to-active-learn methodologies. Through experimental validation, our approach demonstrates promising results, especially in scenarios where there are limited initially labeled instances, particularly when the number of labeled instances per class is extremely limited.