Abstract:

This dataset reports data generated through the investigation of the problem of detection of label-noise in large pattern-recognition datasets. Specifically, results of label-noise reduction on two datasets are reported. The University of California, Irvine (UCI) Letter Recognition dataset and the Mixed National Institute of Standards and Technology (MNIST) Digit Recognition dataset was used to train an algorithm. The algorithm was then tested a Plankton dataset collected by the SIPPER (Shadow Imaging Particle Profiler and Evaluation Recorder) camera imaging system during trips to the site of the Deepwater Horizon Oil Spill. The experiment with the Plankton dataset represented a more practical application of data cleansing, because the label-noise naturally occurred in the data. Data presented in this database include the noise detected for four classes, total noise detected, and the percent accumulative noise detected for both the UCI and MNIST datasets. SIPPER data are not reported in this dataset. SIPPER data can be found in dataset R1.x130.000:0002 "Application of image processing and machine learning techniques to distinguish suspected oil droplets from plankton and other particles for the SIPPER imaging system". On a dataset that contained images of plankton with inadvertent noise, the new algorithm was able to detect all incorrect samples in the class of interest by reviewing only 5% of the data. Thus, the described approach helps to significantly reduce the effort needed to remove label-noise from data. Data published in: Felatyev, S., M. Shreve, K. Kramer, L. Hall, D. Goldgof, R. Kasturi, K. Daly, A. Remsen, H. Bunke. 2012. Label-Noise Reduction with Support Vector Machines. International Conference on Pattern Recognition (ICPR), November 2012, Tsukuba Science City, Japan.

Suggested Citation:

Fefilatyev, Sergiy, et al.. 2015. Label-noise reduction with support vector machines. Distributed by: GRIIDC, Harte Research Institute, Texas A&M University–Corpus Christi. doi:10.7266/N71834DN

Purpose:

Use of SVM to reduce noise in data labeling for pattern recognition software.

Data Parameters and Units:

Data parameters (units are counts or % a indicated in table and below) include: For the UCI Letter Recognition Dataset Iteration Total # SV Noise switches Class ‘H’ SV Class ‘H’ SV detected Class ‘R’ SV Class ’R’ noise SV detected Total noise SV detected % of noise detected (Accum.) for the MNIST Dataset Iteration Total # SV Noise switches (both digits) Class ‘9’ SV Class ‘9’ noise SV detected Class ‘7’ SV Class ‘7’ noise SV detected Total noise SV detected % of noise detected (Accum.)

Methods:

1. Train a binary SVM classifier. If the training set contains multiple classes use a one-versus-one strategy. 2. Collect all examples that were identified as a support vector for each pair. 3. Have an expert validate each of these examples, updating its label if necessary. 4. Repeat Steps 1 - 3 until no further label error is found. The University of California, Irvine (UCI) Letter Recognition dataset was used to train an algorithm. Two letter classes, “H” and “R”, were most confused during classification in a training set of 12,000 instances of 16 features and 26 classes (capital letters from A to Z). We introduced noise at 10%, 20% and 30% into the “H” and “R” classes by switching labels between a pair of instances with these two classes. The Mixed National Institute of Standards and Technology (MNIST) Digit Recognition dataset, containing 60,000 instances, 784 features, and 10 classes (digits from 0 to 9) was also used to train and test the algorithm. Both the test and training sets included 10,000 instances. In this case, the classes “7” and “9” were most confused during training. Therefore in the training set we introduced 10%, 20%, and 30% noise.

Provenance and Historical References:

Felatyev, S., M. Shreve, K. Kramer, L. Hall, D. Goldgof, R. Kasturi, K. Daly, A. Remsen, H. Bunke. 2012. Label-Noise Reduction with Support Vector Machines. International Conference on Pattern Recognition (ICPR), November 2012, Tsukuba Science City, Japan.

Individual Files: