Acoustic scattering by layers of plankton interferes with active and passive sonar systems, which are used by the U.S. Navy for collecting environmental data such as the production of bathymetric maps or target detection. These layers cause transmission loss due to bio-attenuation, create reverberation of sound waves, obscure detection of targets, and produce ambient noise. However, there are currently very few studies on these intermediate trophic levels. A project at the U.S. Naval Research Lab is seeking to develop a more effective technique to determine the location and density of the most common plankton species. Experimental data were collected off the coast of Delaware in 2018 using an In Situ Ichthyoplankton Imaging System (ISIIS). The device recorded shadowgraphs of plankton at 16/17 images s-1 or 2 TB of images every 3.5 hours. Two Convolutional Neural Network models, VGG and IchthyNet, were utilized to identify 24 million plankton data samples. This presentation describes an analysis conducted to evaluate and improve the performance of these two CNN models. For the evaluation, a Python script was created to review the models' labels and determine "True Labels,'' a ground-truth categorization based on visual identifications. A total of 4,595 data samples, distributed relatively evenly among 11 species and 1 "unknown" category, were reviewed. The identifications produced by VGG, IchthyNet, and True Labels were compared using a Random Forest classifier, which labeled 2/3 of the data in bulk at over 95% confidence. Results suggested that the CNN models performed best together, with highest accuracies seen when the models agreed on a label. With the True Label dataset, the accuracies ranged between 95.0%-100%. The highest accuracies for the models individually with this dataset were 88.6% for IchthyNet and 87.0% for VGG. For the Random Forest dataset, all species labels had a 100% accuracy rate when IchthyNet and VGG agreed. With this smaller sample size, plots of model output do not show extreme patterns, but they do illustrate a balanced and even set of data with a low bias in terms of number and location. To fully validate the accuracy of the Random Forest results, a greater quantity of data needs to be labeled, and data collection must be extended to additional and more distant locations. DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited.