Automated monitoring and evaluation systems for plant phenotyping are one of the keys to advance and strengthen crop breeding programs. In this study, the improvements of the camera-based sensor system and a weather station from a previous study-assembled mainly from Raspberry Pi products-board with dual cameras (RGB and NoIR) providing high spatial and temporal resolution data-is outlined. Hardware for the internet connection and the power supply system of the sensor were upgraded. Previously, the sensor could automatically capture plant images following user-defined time points; thus, an image processing algorithm (edge computing) was developed and installed to extract digital phenotypic traits from the images after capturing process. With the development, the new sensor system could be integrated with the internet, and a cloud server was configured to store data online (digital traits and raw images). A real-time monitoring system was created to visualize the time series data of a trait development and plant images throughout the season. With such a system, plant breeders will be able to monitor multiple trials for timely crop management and decision-making process, which is also resources efficiency.

The automatic, and accurate plant phenotyping plays important role to improve the crop yield through enabling efficient plant analysis and plant breeding studies. The 3d deep learning allows automatic segmentation of plant parts from point cloud data. However, the network architecture is designed manually and performance is limited to prior experience. The aim of this study is to search for optimal 3d deep networks to perform the plant part segmentation. We perform the 3d neural architecture search by training a super network composed of candidate networks. Using the trained super network, the evolutionary searching is used to search for top performing architecture. The results demonstrate the searched architecture outperforms manually designed architectures by attaining mean IoU and accuracy of more than 90% and 96%, respectively. The searched architecture achieves more than 83% class-wise IoU for all main stem, branches, and boll class. This plant part segmentation method shows promising results and holds potential to be utilized by plant breeders for enhancing the production quality.

Hyperspectral imaging is a promising method to predict traits in a high-throughput manner with the potential to unlock quantitative genetic studies. Researchers have successfully modeled physiological traits such as vegetative Nitrogen content, but scope of methodology and lack of truly novel testing data hinder large scale trust in the process. Here, I explore the ability to model leaf Nitrogen content from hyperspectral reflectance data collected with a LeafSpec imaging device on 22 maize hybrids. Three broad strategies based on different input feature sets are undertaken. Strategy one mines data for the most informative hyperspectral channels and then constructs a normalized index similar to NDVI as input features. Strategy two considers all 364 channels of hyperspectral data and makes predictions using various machine learning techniques; partial least squares regression(PLSR), random forest regression, and a feed-forward neural net regression. Strategy three aims to take advantage of the spatial distribution of hyperspectral data on the leaf surface by training a convolutional neural net(CNN). A normalized visual index constructed from bands most correlated with nutrient content out-performed established NDVI. PLSR was the most accurate algorithm, followed by feed-forward neural net and then CNN, based on coefficient of determination score. PLSR is well established as a robust method for hyperspectral prediction which is further evidenced by this study. This is one of the first applications of CNN for hyperspectral data. Despite not being the most accurate algorithm there remains room for hyper-parameter optimization.

An organism's phenome results from expression of its genome (nature) under certain environment and management effects (nurture) and interactions between these factors, as well as measurement error. For over 30 years, DNA sequencing and genomics tools advanced to where it's now feasible to saturate genomes of segregating individuals, such that polymorphisms at nearly any position can be determined from other known positions. This is due to structure, linkage disequilibrium (LD), or linkage and is a powerful tool for genomic prediction and investigating biological phenomena. In contrast, most phenomics to date focuses on automating previously known "traits" as measurable and interpretable phenotypes; akin to focusing on measuring a single DNA marker rather than measuring an entire saturated genome. Viewing phenomics as a platform for discovery, similar to genomics, opens new methods for capturing phenomena in nature and nurture. Saturating a phenome would mean that an individual's fitness, performance, responses to environment and/or specific phenotypes could be accurately predicted in untested environments. To date, our experience with phenomic prediction for cumulative, complex phenotypes such as grain yield suggests it's possible to predict organismal performance in untested environments, possibly better than genomic methods despite less advanced tools and data. Factors limiting to saturating a phenome are evaluating enough individuals and environments, but more importantly, tools and methods to extract or "sequence" more phenomic features. Successfully saturating phenomes will impact every aspect of science and society, in biological disciplines from germplasm curators, physiologists to breeders, to education, the courtroom and policy.

Unmanned aerial vehicle (UAV)-based imagery has become widely used in collecting agronomic traits, enabling a much greater volume of data to be generated in a time-series manner. As one of the cutting-edge imagery analysis tools, machine learning-based object detection provides automated techniques to analyze these imagery data. In our previous study, UAVs have been used to collect aerial photography for field trials of 233 diverse inbred lines, grown under different nitrogen treatments. Images were collected during different plant developmental stages throughout the growing season. This dataset of images has here been used in developing machine learning techniques to obtain automated tassel counts at the plot level through the season. To improve detection accuracy, we have developed an image segmentation method to remove non-tassel pixels and then feed these filtered images into machine learning algorithms. As a result, our method showed a significant improvement in the accuracy of maize tassel detection. This method can be used in future research to produce time-series counts of tassels at the plot level, and will allow for accurate estimates of flowering-related traits, such as the earliest detected flowering date and the duration of each plot's flowering period. This phenotypic data and the trait-associated genes provide new opportunities for crop improvement and to facilitate future plant breeding.

Imaging of plants using multi-camera arrays in high-density growth environments is a strategy for affordable high-throughput phenotyping. In multi-camera systems, simultaneous imaging of hundreds to thousands of plants eliminates the time delay in measurements between plants seen in plant-to-camera or camera-to-plant systems, which allows for the analysis of plant growth, development, and environmental responses at a high temporal resolution. On the other hand, high plant density, camera-to-camera variation, and other trade-offs increase the complexity of data analysis. Here we present two recent updates to the PlantCV image analysis package to improve usability when working with multi-plant datasets. First, we introduce a method to automate detection of plants organized in a grid layout, reducing the need to make separate workflows for each camera in a multi-camera system. Second, we reduced the number of input and output parameters for functions handling the shape and location of plants and introduce automatic iteration over multiple objects of interest (e.g. plants), reducing the level of programming needed to build workflows.

Image based high throughput plant phenotyping is a powerful tool to capture and quantify diverse plant traits. The available commercial platforms are often cost-prohibitive. This study describes the development of a low cost, automated plant phenotyping platform, which can acquire images, transfer data, segment the images, extract the traits and perform data analysis using low-cost microcomputers, cameras and IoT irrigation system. Quantifiable plant traits (e.g., shape, area, height, color) were extracted from the plant images using an in-house pipeline developed in R language. An experiment of water stress (waterlogging and drought) on Mentha arvensis (Menthol mint) crop (cv. CIM-Kosi) was conducted to demonstrate image traits being used as a proxy for plant response to water stress. It was found that the effect of drought stress on plant height and number of secondary branches could be correlated to color traits of plant canopy images. Also, the effect of waterlogging stress on chlorophyll and flavonoid content could be related to the shape traits of plant canopy images and effect on waterlogging on plant height and canopy width could be associated with color and texture traits. The imaging platforms could successfully demonstrate a viable low-cost solution for incorporating high-throughput plant phenotyping in various plant stress related research applications.

3D scans of real world objects are often represented by point clouds, creating XYZ-coordinates of individual scan points. However, unlike point clouds that are generated from CAD data, points generated from a real world scene lack information about their local context, making segmentation of the structural information contained in the data difficult. Using neural networks (e.g. PointNet) has shown promising results. However, this approach is not well suited for scans of large areas of similar objects, like e.g. a wheat field, because of limitations of the input vector size of the neural network. In addition, point clouds are often unordered, further complicating processing. Since point clouds of biological objects often contain recurring features, we propose to subdivide the point cloud into locally neighboring subsets with a fixed number of points. The collection of subsets can then be used to train neural networks. This approach preserves the original resolution of the point cloud while offering simple data augmentation concepts like creating a number of different subset collections from the same ground truth. There are several advantages to this approach, like significantly simplifying the training phase, because a single, large annotated scan can be sufficient for training, utilizing the similarity of the instances of a plant in the field.

Henry Cordoba-Novoa1, Isabella Chiaravalotti1 , Laura Esquivel1, Robert McGee1, Jeff Smith1, Shangpeng Sun1 and Valerio Hoyos-Villegas1

We investigate the robustness of 3D point-based deep learning for organ segmentation of 3D plant models against varying reconstruction quality of the surface. The reconstruction quality is quantified in two ways: 1) The number of acquisitions for partial 3D scans and 2) the amount of noise. High quality models of real rosebush plants are used to collect point clouds in a controlled simulation environment as a way to degrade surface quality systematically. We show that the well-known 3D point-based neural network PointNet++ is capable of operating effectively on low quality and corrupted data for the task of plant organ segmentation. The results indicate that investing on developing deep learning methods has the potential of advancing applications of automated phenotyping, especially for low-quality 3D point clouds of plants. Keywords: plant phenotyping, organ segmentation, robustness analysis, point-based deep learning (a) (b) Figure 1: A 3D rosebush model from ROSE-X data set: (a) point cloud; (b) triangular mesh model.