Methods:
A. SIPPER Field Deployments The SIPPER camera imaging system (Fig. 1) was developed by the University of South Florida, Center for Ocean Technology and described in Samson et al. (2001) and Remsen et al. (2004). The towed platform carried several environmental sensors (CTD, oxygen, chlorophyll fluorescence, transmissometer: see Supplemental Information- Instruments). SIPPER used a high speed Dalsa Piranha-2 line-scan camera and a pseudo-collimated LED generated light sheet to image the shadows and outlines of resolvable particles that passed through a 100 cm2 field of view. The operational optical resolution of the system is ~65 m. SIPPER was towed at speeds between 2-3 knots in an oblique profile through the water column, spending approximately equal amounts of time at each meter of depth between the surface and 300 m. At stations with a bottom depth shallower than 300 m, SIPPER was towed within approximately 5 m from the seafloor. Imaging and environmental data were stored internally on a Firewire hard drive and processed upon retrieval of the SIPPER instrument from a deployment using a customized software package called the Plankton Image Classification and Extraction Software (PICES). PICES was used to extract images of interest, classify them using user-specified training libraries and to manage the SIPPER images and environmental data collected. Currently, the PICES manages information for over 137 million SIPPER images. B. SIPPER Data Analyses SIPPER data, including concurrently collected environmental instrument data, were offloaded to a desktop PC after every deployment. Using the Plankton Instrument Classification and Extraction Software (PICES) (Kramer et al. 2011), images greater than or equal to 250 pixels in total area (~0.5 mm equivalent spherical diameter or ESD) were extracted from the raw SIPPER data, preliminarily classified and automatically entered into a MYSQL based image database. A custom database management application called PICES Commander was used to manage and classify the resulting dataset. Classification of SIPPER images involved several steps including both automated classification and manual labeling of the images. (1) Image Extraction and Preliminary Classification: During the image extraction step, features for each image are determined and entered into a feature table in the PICES database and an initial classification for each image is predicted using a comprehensive, multi-class feature selection (MFS) support vector machine (SVM) (Kramer et al., 2011). This classifier was initially developed from training images collected in the Gulf of Mexico prior to the Deepwater Horizon oil spill. Training images and new image classes were added throughout the sampling period and slowly replaced the images that were collected before the spill. We utilized a hierarchical naming format for the particle and plankton image classes to reflect the level of similarity between image classes (Fig.1). More closely related image classes shared one or more parts of their name separated by underscores and this naming schema is used in the classification effort. (2) Manual Validation: After image extraction and the first run of the comprehensive classifier, thumbnail images of the resulting classifications were then examined using the image browsing and validation capability of PICES Commander. Image classes that consisted of numerous true positive examples (for example calanoid copepods or detritus) were noted for inclusion in the final classification effort and then ignored while image classes that were less abundant or rare were closely examined. True positive image examples from these rarer image classes were validated, meaning that their image class label could not be relabeled during a subsequent classification. This was done for every predicted image class until all rare images were validated from the first comprehensive classification. Occasionally, a new image class was encountered. When this occurred, that class was added to the comprehensive classifier and any encountered examples would be validated and added to the trai! ning library (3) Run Final Comprehensive MFS-SVM Classification: After all deployments had their images extracted, initially classified, and new image classes encountered, labeled, and added to the training library, the final comprehensive MFS-SVM classifier was rebuilt and rerun on all deployments using all the new training examples that had been uncovered in Step 2. The purpose of this classification cycle was to identify which common classes to include in the ultimate dual classifier, identify deployments where less common image classes might be abundant enough to include in the ultimate dual classifier, and validate any new rare image examples that may have been identified during the second run of the comprehensive classifier. (4) Build Deployment-Specific MFS-SVM Classifiers: For each deployment, after rare and uncommon groups were searched for true positives and validated in Step 3, a MFS_SVM classifier was pared down to the most common classes found in that deployment and rebuilt. New MFS classifiers were, therefore, built for each deployment with classes specific to that deployment. Multiple deployments could share the same final MFS classifier if they shared the same common groups. A final feature selection was then run for all the image classes in the final MFS Classifiers, using Amazon High Performance Computing, to create the most appropriate feature set for the classifiers. (5) Build Deployment-Specific BFS-SVM Classifiers: Another set of SVM classifiers was also built using binary feature selection (BFS) as described in Kramer et al. (2011), where features and parameters are tuned for each class pair combination. These BFS classifiers used the same image classes as the MFS classifiers so that they were specific for the particular conditions encountered during each deployment. By using pair-wise feature selection, overall classification accuracy can be improved, and feature selection and SVM training time can be reduced. (6) Run Dual Classification: Lastly, these two sets of classifiers (MFS-SVM and BFS-SVM) were used to run a modified version of the dual classifier described for the video plankton recorder (VPR) by Hu and Davis (2006). As the name implies, a dual classifier makes use of two classifiers where each classifier makes its own separate prediction and a particle image is only labeled as a specific image class if both classifiers agree, otherwise the particle image is labeled “other”. In the case of the VPR, a SVM classifier using texture based features and a neural net using shaped-based features were used as the two classifiers. For the SIPPER data, a neural net performed poorly and it was determined that a SVM BFS using separate feature sets for each pairwise classification was the best alternative. Along with the SVM MFS classifier, the SVM BFS made up the SIPPER dual classifier. A further modification took advantage of the hierarchical naming structure of SIPPER image cla! sses by allowing for predictions from partial matches between the two classifiers. For example if one classifier predicted that an unknown image was crustacean_copepod_calanoid and the other classifier predicted it was crustacean_copepod_oithona, it was very likely that the image was in fact a copepod. Rather than labeling this disagreement as “other” as the traditional dual classifier would have done, our modified classifier would label the image as crustacean_copepod since both classifiers agreed to that partial match. Only when the two classifiers predictions shared no common root in the SIPPER class names (e.g. one classifier predicts an unknown image is detritus_snow and the other predicts larvacean) would an image be labeled “other”. Dual classifiers are used because they improve specificity (reduce the rate of false positives) and thereby give more accurate abundance estimations, especially in regions of low relative abundance. Dual classifiers were run on ea! ch deployment using the most common image classes in that deployment as determined in Step 3. A total of 32 individual image classes were common enough to be used in at least one deployment and are listed below. Validated images from rare and uncommon classes that were removed from the classifier retained their validated labels in the classifier output. C. SIPPER Data Files SIPPER and environmental data are reported by cruise in excel files. Each deployment is reported in a separate tab in cruise files. Every SIPPER deployment data sheet shows the same set of classes in order to evaluate the spatial and temporal variability of image classes. Zero abundances in any class means that no images were observed for those classes. The SIPPER data are from the downcast, unless otherwise noted. Environmental data are reported to the far right of the SIPPER abundance data in the excel spreadsheet. D. SIPPER Image Class Descriptions There are several types of images classes: (1) dual classifier classes, (2) rare or uncommon classes, and (3) other image classes resulting from the dual classification. Examples of images for these classes are available on http://www.marine.usf.edu/zooplankton/. (1) Dual Classifier Classes. The following training library classes were used in at least one of the final dual classifications. These 25 classes had numerous images in at least one or more deployments. These were the more common classes that made up about 90% of the images. Chaetognath: Chaetognaths of various genera found in the Gulf of Mexico Crustacean_cladoceran_Evadne: The cladoceran species Evadne tergestina Crustacean_cladoceran_Penilia: The cladoceran species Penilia avirostris Crustacean_copepod_calanoid: All calanoid copepod species Crustacean_copepod_Macrosetella: Harpactacoid genera Macrosetella spp. and Microsetella spp. Crustacean_copepod_nauplii: Assorted copepod nauplii (larval stages) Crustacean_copepod_Oithona: Cyclopoid copepod genus Oithona spp. Crustacean_copepod_poecilostomatoid: Poecilostomatoid copepod genera include Oncaea, Corycaeus, Farranula and others Crustacean_eumalocostracan_Lucifer: The sergestid shrimp genera Lucifer Crustacean_ostracod: Ostracods Detritus_blob: Unknown detrital particles having a characteristic shape Detritus_snow: Detrital aggregates and marine snow Echinoderm_bipinnaria; Auricularia and bipinnaria larval stages of echinoderms Echinoderm_plutei: Pluteus larval stages of echinoderms Gelatinous_ctenophore: This class is comprised of the various species of ctenophores Gelatinous_hydromedusae_other: Assorted species of hydromedusae, narcomedusae and scyphomedusae Gelatinous_hydromedusae_small: Small unidentifiable hydromedusae Gelatinous_siphonophore: Various forms of siphononophora Gelatinous_tunicate_doliolid: Various life history stages of doliolids Gelatinous_tunicate_salps: Individual and colonial forms of tunicate salps Larvacean: Larvaceans, with and without their mucous house Mollusc_pteropod_conical: Cone-shaped thecosome pteropod species including Calvolinia, Clio, Creseis, and Styliola species Other: Unidentified particles and dual classifier disagreements are labeled as this class Phytoplankton_diatom centric: Centric forms of various diatom species Phytoplankton_diatom coil: Coiled chains of various diatom species Phytoplankton_strands. Elongate phytoplankton chains and filaments Phytoplankton_Trichodesmium: Forms of the colonial cyanobacteria Trichodesmium Protist_acantharia: Solitary acantharia and other conspicuously spined protists Protist_knobby: Unidentified radiolarian with ovoid, granular body adorned with knobby spines Protist_Noctiluca: Various life history stages of Noctiluca sp. dinoflagellates and related dinoflagellate genera Protist_radiolarian: Ovoid, short-spined radiolarians Protist_unknown: Unidentified and uncommon protists (2) Rare or uncommon classes. The following classes were rare or uncommon classes used in the comprehensive MFS classifier run (Step 3). These classes were all manually validated. This process was done to identify and label true positive examples of rare and uncommon image classes before running the final dual classification, which only identified common classes. Rare and uncommon classes included the following: Crustacean_copepod_Copilia: This class is comprised of individuals of the dorsally flattened poecilostomatoid copepod genera Copilia Crustacean_copepod_Macrosetella_tricho: This class is comprised of individuals of the harpacticoid copepod genus Macrosetella, which is associated with the colonial cyanobacteria Trichodesmium Crustacean_copepod_Sapphirina: This class is comprised of examples of the dorsally flattened and iridescent poecilostomatoid genus Sapphirina Crustacean_eumalacostracan_euphausiid: This class includes adult and juvenile stages of euphausiids and mysids Crustacean_eumalacostracan_other: This class includes amphipods, shrimp, stomatopods, zoea (crab larvae) and other eumalocostracan groups Crustacean_phyllosome: This class is comprised of examples of lobster phyllosome larvae. These are flattened forms that often are associated with gelatinous organisms Detritus_molts: This class includes carcasses of dead zooplankton and molts from crustaceans Fish: This class is comprised of various larval forms of fish from the Gulf of Mexico Fish_egg: This class is comprised of fish eggs Lancelet: This class is comprised of lancelets or amphioxi, fish-like marine chordates of the order Amphioxiformes Larvae_polychaete: This class is comprised of the larval forms of polychaete worms Larvae_tornaria: This class is comprised of the larval planktonic tornaria stage of hemichordates or acorn worms Larvae_veliger: This class is comprised of the larval planktonic veliger stage of molluscs, such as snails and bivalves Mollusc_Atlanta: This class is comprised of individuals of the marine gastropod mollusk family Atlantidae including the species Atlanta and Oxygyrus Mollusc_heteropod: This class includes various species of heteropod imaged by SIPPER in the Gulf of Mexico Mollusc_pteropod Mollusc_pteropod_gymnosome: These are pelagic opisthobranch molluscs that are often predatory on other pteropod groups. They include the genera Clione, Clionopsis, Thliptodon, and many others Mollusc_pteropod_pseudothecosomata: These are individuals of the pseudothecosomata, an infraorder of the thecosomata pteropods. These include species such as Cymbulia, Desmopteris, Gleba, Corolla and many others Polychaete: This class is comprised of annelid and polychaete worms imaged by SIPPER Protist_Collozoum: This class is comprised of species of the colonial radiolarian genus Collozoum and other genera within the order Spumellaria that form small to large colonies of individuals by extending their cytoplasm to form an anastomosing web of pseudopodia enclosed within a common gelatinous matrix Protist_electric:This class is comprised of an unknown deep water (>100 m depth) protist with long filamentous pseudopodia that are used to catch particles and plankton Protist_Thalassicola: This class is comprised of the radiolarian genus Thalassicola (3) Other Image Classes. There are several image classes that only result from running the dual classifier using a hierarchical naming schema. The image classes below are populated by images that result from a disagreement between the two classifiers in the dual classifier where the two guesses share part of the class name in common. These images are then classified to the level of agreement within the naming structure. An example might be when a classifier could not agree between crustacean_ostracod and crustacean_cladoceran_evadne. Both guesses share the 1st order root crustacean in common, so the dual classifier would classify such an image as crustacean. These classes include the following: Crustacean: Results from disagreements between crustacean_ groups. Crustacean_copepod: Results from disagreements between crustacean_copepod groups. Detritus: Results from disagreements between detritus_snow and detritus_blob. Gelatinous zooplankton: Results from disagreements between gelatinous_ groups such as gelationous_siphonophore and gelatinous_tunicate_doliolid. Phytoplankton: Results from disagreements between phytoplankton groups. Protist: Results from disagreements between protist groups. E. References Hu, Q. and Davis, C. 2006. Accurate automatic quantification of taxa-specific plankton abundance using dual classification with correction. Marine Ecology Progress Series 306: 51-61. Kramer, K., Goldgof, D., Hall, L., Remsen, A. 2011. Increased classification accuracy and speedup through pair-wise feature selection for support vector machines. IEEE Symposium Series on Computational Intelligence Proceedings, 318-324. Remsen, A., Hopkins, T.L., and Samson, S. 2004. What you see is not what you catch: a comparison of concurrently collected net, Optical Plankton Counter, and Shadowed Image Particle Profiling Evaluation Recorder data from the northeast Gulf of Mexico. Deep Sea Research I 51: 129-151. Samson, S., Hopkins, T., Remsen, A., Langebrake, L., Sutton, T., Patten, J., 2001. A system for high resolution zooplankton imaging. IEEE Journal of Oceanic Engineering 26 (4), 671–676.
View Metadata