Discussion
In this study, we identified hybrids between native Japanese giant
salamanders and non-native Chinese giant salamanders from images using
deep learning. Historically, visual screening by experts and DNA
analysis were applied to identify hybrids. However, the scarcity of
experts and the time and cost of DNA analysis were barriers to effective
screening. Therefore, we have proposed a novel approach to identifying
hybrids using an image recognition technique. A total of 6 native and 15
hybrid individuals were used, and all were correctly classified by the
AI model with an accuracy of 100% in our experimental setting.
Furthermore, highlighted regions that affect the AI model’s prediction
suggested that the model distinguished between native and hybrid species
based on spot patterns. Although deep learning has already been applied
to species and individual identification, this is the first study we
know that identifies hybrid species.
EfficientNet-V2 demonstrated that head spot patterns could be used to
identify native and hybrid species. One reason for successfully
classifying all individuals is the quality of the training and test
images. In this study, photographs were taken from a short distance;
thus, the high accuracy can be attributed to the clear spot patterns in
the images. Another reason is that the salamanders’ heads were
photographed from a similar angle (Supplementary Figure 3). For example,
previous studies have shown that different photo angles reduce
identification accuracy (Arzoumanian et al. 2005). In this research, we
photographed all individuals from directly above. These images used for
training and testing facilitated the comparison of spot patterns and
ensured highly accurate results. Training and test images obtained on
the same day could also have influenced the high performance. In the
future, our approach performance should be carefully evaluated in a
varied environment, using images from different dates and locations,
before implementing this technology in the field.
Visualized distribution of the heat maps was different for native and
hybrid species. For the native species, the model focused on distinctive
and large black spots, while for the hybrids, it focused on the pale and
ambiguous wide region. These results suggest that the differing spot
patterns between native and hybrid species can be utilized for
classification. In general, native species have distinctive and large
black spots (Supplementary Figure 1), whereas the spots of hybrids are
more indistinct than those of native species (Supplementary Figure 2).
Experts use these spot pattern differences as one of the criteria to
identify hybrid individuals. The results of this study revealed that
deep learning would distinguish between native and hybrid species using
the same pattern recognition as experts. The heat map could be used as
an instruction guide for the general public on hybrid identification
because the highlighted graphical figures are visually comprehensible.
Although our approach has achieved high accuracy in identifying native
species and hybrids in this study, several challenges still exist.
Firstly, we did not consider the hybridization degree, which affects the
spot pattern in hybrids. The hybrid captured in Hiroshima used in this
study was found in the river recently, which suggests that the
generation is less advanced. Since hybrid individuals between Japanese
and Chinese giant salamanders are fertile, the spot pattern varied
depending on several factors, such as generation. Future work should
examine the relationship between the degree of genetic introgression and
identification accuracy. Secondly, combining this method with DNA
analysis is essential because deep learning-based identification has
limitations. For example, due to hybridization, some hybrids have
previously been observed with spots indistinguishable from those of
Japanese giant salamanders. DNA analysis is the only method to determine
the species in such cases. Therefore, our technology could be applied
for the early detection of suspected hybrids through citizen science and
quick identification by computer vision. In addition, advanced research
might allow the identification of backcrossed hybrids that are difficult
to distinguish even for experts because spots are extremely close to
native species. Finally, this study was conducted in the daytime in
uniform photographic conditions. Giant salamanders must be photographed
under lights in field-based surveys because they are nocturnal. In the
future, it is necessary to determine whether images obtained under
various light conditions could be used to identify hybrids.
Hybridization between native and invasive species is one of the major
causes of biodiversity loss (Bourret et al. 2022). However, detecting
hybrids was challenging when the hybrids were similar to the native
species. Deep learning image recognition techniques can be a valuable
tool to support the visual identification of hybrids. We proposed a new
approach for classifying native species and hybrids using smartphone
images that could be utilized in citizen science. Hybrid identification
based on spot patterns has previously been difficult and thus limited to
experts; however, artificial intelligence analysis allows the public to
detect hybrids easily. In particular, the distribution of hybrids is
expanding, meaning that managing hybrids is a priority task for
conserving Japanese giant salamanders. The findings of this study can
potentially prevent the future spread of hybrids by providing a method
for the efficient discovery of these individuals.