This paper presents an autonomous robot capable of picking strawberries continuously in polytunnels. Robotic harvesting in cluttered and unstructured environment remains a challenge. A novel obstacle‐separation algorithm was proposed to enable the harvesting system to pick strawberries that are located in clusters. The algorithm uses the gripper to push aside surrounding leaves, strawberries, and other obstacles. We present the theoretical method to generate pushing paths based on the surrounding obstacles. In addition to manipulation, an improved vision system is more resilient to lighting variations, which was developed based on the modeling of color against light intensity. Further, a low‐cost dual‐arm system was developed with an optimized harvesting sequence that increases its efficiency and minimizes the risk of collision. Improvements were also made to the existing gripper to enable the robot to pick directly into a market punnet, thereby eliminating the need for repacking. During tests on a strawberry farm, the robots first‐attempt success rate for picking partially surrounded or isolated strawberries ranged from 50% to 97.1%, depending on the growth situations. Upon an additional attempt, the pick success rate increased to a range of 75–100%. In the field tests, the system was not able to pick a target that was entirely surrounded by obstacles. This failure was attributed to limitations in the vision system as well as insufficient dexterity in the grippers. However, the picking speed improved upon previous systems, taking just 6.1 s for manipulation operation in the one‐arm mode and 4.6 s in the two‐arm mode.

An autonomous strawberry???harvesting robot: Design, development, integration, and field evaluation

https://ro.ecu.edu.au/cgi/viewcontent.cgi?article=6091&context=ecuworkspost2013

A comprehensive review of fruit and vegetable classification techniques

This work presents a machine vision system for the localization of strawberries and environment perception in a strawberry-harvesting robot for use in table-top strawberry production. A deep convolutional neural network for segmentation is utilized to detect the strawberries. Segmented strawberries are localized through coordinate transformation, density base point clustering and the proposed location approximation method. To avoid collisions between the gripper and fixed obstacles, the safe manipulation region is limited to the space in front of the table and underneath the strap. Therefore, a safe region classification algorithm, based on Hough Transform algorithm, is proposed to segment the strap masks into a belt region in order to identify the pickable strawberries located underneath the strap. Similarly, a safe region classification algorithm is proposed for the table, to calculate its points in 3D and fit the points onto a 3D plane based on the 3D point cloud, so that pickable strawberries in front of the table can be identified. Experimental tests showed that the algorithm could accurately classify ripe and unripe strawberries and could identify whether the strawberries are within the safe region for harvesting. Furthermore, harvester robot's optimized localization method could accurately locate the strawberry targets with a picking accuracy rate of 74.1% in modified situations.

/pdf/fruit-localization-and-environment-perception-for-strawberry-50gwr2donb.pdf

Fruit Localization and Environment Perception for Strawberry Harvesting Robots

Measurement of plant characteristics is still the primary bottleneck in both plant breeding and crop management Rapid and accurate acquisition of information about large plant populations is critical for monitoring plant health and dissecting the underlying genetic traits In recent years, high-throughput phenotyping technology has benefitted immensely from both remote sensing and machine learning Simultaneous use of multiple sensors (eg, high-resolution RGB, multispectral, hyperspectral, chlorophyll fluorescence, and light detection and ranging (LiDAR)) allows a range of spatial and spectral resolutions depending on the trait in question Meanwhile, computer vision and machine learning methodology have emerged as powerful tools for extracting useful biological information from image data Together, these tools allow the evaluation of various morphological, structural, biophysical, and biochemical traits In this review, we focus on the recent development of phenomics approaches in strawberry farming, particularly those utilizing remote sensing and machine learning, with an eye toward future prospects for strawberries in precision agriculture The research discussed is broadly categorized according to strawberry traits related to (1) fruit/flower detection, fruit maturity, fruit quality, internal fruit attributes, fruit shape, and yield prediction; (2) leaf and canopy attributes; (3) water stress; and (4) pest and disease detection Finally, we present a synthesis of the potential research opportunities and directions that could further promote the use of remote sensing and machine learning in strawberry farming

Remote Sensing and Machine Learning in Crop Phenotyping and Management, with an Emphasis on Applications in Strawberry Farming

Background Shape is a critical element of the visual appeal of strawberry fruit and is influenced by both genetic and non-genetic determinants. Current fruit phenotyping approaches for external characteristics in strawberry often rely on the human eye to make categorical assessments. However, fruit shape is an inherently multi-dimensional, continuously variable trait and not adequately described by a single categorical or quantitative feature. Morphometric approaches enable the study of complex, multi-dimensional forms but are often abstract and difficult to interpret. In this study, we developed a mathematical approach for transforming fruit shape classifications from digital images onto an ordinal scale called the Principal Progression of k Clusters (PPKC). We use these human-recognizable shape categories to select quantitative features extracted from multiple morphometric analyses that are best fit for genetic dissection and analysis. Results We transformed images of strawberry fruit into human-recognizable categories using unsupervised machine learning, discovered 4 principal shape categories, and inferred progression using PPKC. We extracted 68 quantitative features from digital images of strawberries using a suite of morphometric analyses and multivariate statistical approaches. These analyses defined informative feature sets that effectively captured quantitative differences between shape classes. Classification accuracy ranged from 68% to 99% for the newly created phenotypic variables for describing a shape. Conclusions Our results demonstrated that strawberry fruit shapes could be robustly quantified, accurately classified, and empirically ordered using image analyses, machine learning, and PPKC. We generated a dictionary of quantitative traits for studying and predicting shape classes and identifying genetic factors underlying phenotypic variability for fruit shape in strawberry. The methods and approaches that we applied in strawberry should apply to other fruits, vegetables, and specialty crops.

/pdf/multi-dimensional-machine-learning-approaches-for-fruit-5319j1xwdv.pdf

Multi-dimensional machine learning approaches for fruit shape phenotyping in strawberry.

. Shape is one of the most important traits of agricultural products due to its relationships with the quality, quantity, and value of the products. For strawberries, the nine types of fruit shape were defined and classified by humans based on the sampler patterns of the nine types. In this study, we tested the classification of strawberry shapes by machine learning in order to increase the accuracy of the classification, and we introduce the concept of computerization into this field. Four types of descriptors were extracted from the digital images of strawberries: (1) the Measured Values (MVs) including the length of the contour line, the area, the fruit length and width, and the fruit width/length ratio; (2) the Ellipse Similarity Index (ESI); (3) Elliptic Fourier Descriptors (EFDs), and (4) Chain Code Subtraction (CCS). We used these descriptors for the classification test along with the random forest approach, and eight of the nine shape types were classified with combinations of MVs + CCS + EFDs. CCS is a descriptor that adds human knowledge to the chain codes, and it showed higher robustness in classification than the other descriptors. Our results suggest machine learning's high ability to classify fruit shapes accurately. We will attempt to increase the classification accuracy and apply the machine learning methods to other plant species.

/pdf/classification-of-strawberry-fruit-shape-by-machine-learning-5cf5hq9p6h.pdf

Classification of strawberry fruit shape by machine learning

Fruit shape of cultivated strawberry (Fragaria × ananassa Duch.) is an important breeding target. To detect genomic regions associated with this trait, its quantitative evaluation is needed. Previously we created a multi-parent advanced-generation inter-cross (MAGIC) strawberry population derived from six founder parents. In this study, we used this population to quantify fruit shape. Elliptic Fourier descriptors (EFDs) were generated from 2 969 two-dimensional binarized fruit images, and principal component (PC) scores were calculated on the basis of the EFD coefficients. PC1-PC3 explained 96% of variation in shape and thus adequately quantified it. In genome-wide association study, the PC scores were used as phenotypes. Genome wide association study using mixed linear models revealed 2 quantitative trait loci (QTLs) for fruit shape. Our results provide a novel and effective method to analyze strawberry fruit morphology; the detected QTLs and presented method can support marker-assisted selection in practical breeding programs to improve fruit shape.

Strawberry fruit shape: quantification by image analysis and QTL detection by genome-wide association analysis

/pdf/genome-wide-association-study-of-strawberry-fruit-quality-2veg3vv284.pdf

Genome-wide Association Study of Strawberry Fruit Quality-related Traits Using a MAGIC Population Derived from Crosses Involving Six Strawberry Cultivars

Male sterility is defined as the loss of pollen fertility, and it represents a plant reproductive isolation symptom, along with self-incompatibility. It plays an important role in the efficient production of F1-hybrid seeds, which results in affordable seed prices for farmers. Male sterile cultivated strawberry Fragaria × ananassa Duch. plants were found in an F1 population and reciprocal backcrossed populations derived from a cross between ‘Fukuoka S6’ and ‘Kaorino’. Male sterile plants were clearly distinguished from male fertile plants in those populations based on the anther color. The pollen of the male sterile plants was a lighter yellow color and not maturely shaped compared with pollen of male fertile plants. Genotyping was performed using EST-SSR markers in the three populations. Quantitative trait locus analyses for pollen fertility were conducted independently using three kinds of populations, and this revealed that male sterility was controlled by three independent chromosomal regions in these populations, which corresponded to chromosome 4 in the wild strawberry (Fragaria vesca) genome. One region was derived from ‘Fukuoka S6’ and the other two regions from ‘Kaorino’. The segregation patterns of fertile and sterile plants in each population clearly supported the three gene theory of male sterility in cultivated strawberries. The accumulation of recessive alleles at the three regions led to male sterility, and the existence of a dominant allele in at least one region resulted in fertile pollen. Male sterile plants were also found in two self-pollenated populations derived from ‘Fukuoka S6’ and ‘Kaorino’, and the effects of the three regions were validated. The adaptability levels of the three genes with different genetic backgrounds were also evaluated using core collection cultivars and selected lines derived using recurrent selection. We also detected flanking DNA markers for the three regions associated with male sterility. The use of these markers, which are in the vicinity of quantitative trait loci and responsible for malesterility, could increase the efficiency of producing seed-propagated strawberry F1-hybrids.

Detection of Chromosomal Regions for Male Sterility in the Cultivated Strawberry Fragaria × ananassa Duch.

Male sterility is one of the reproductive isolation systems in plants and quite useful for F1 seed production. We previously identified three independent quantitative trait loci (QTLs) for male sterility of cultivated strawberry, Here, we identified the specific subgenomes in which these QTLs are located by QTL-seq approach. QTLs qMS4.1, qMS4.2, and qMS4.3 were mapped separately in subgenomes Fvb4-4, Fvb4-3, and Fvb4-1, respectively, in ‘Camarosa’ genome assembly v. 1.0.a1. Candidate regions of qMS4.1 and qMS4.3 were clearly detected around 12–26 Mb in Fvb4-4 and 12–14 Mb in Fvb4-1, respectively; those of qMS4.2 were fragmented in Fvb4-3, which suggests that some scaffolds were incorrectly assembled in Fvb4-3. qMS4.3 was mapped to chr4X1 of ‘Reikou’ genome assembly r2.3, and qMS4.1 and qMS4.2 were both mapped to chr4Av, which indicates that differentiation of the subgenomes in which both QTLs are located was insufficient in ‘Reikou’ r2.3. Although ‘Camarosa’ genome assembly v. 1.0.a1 is an unphased map, which merges homologous chromosomes into one sequence, ‘Reikou’ genome assembly r2.3 is a phased map, which separates homologous chromosomes. QTL mapping to different reference genomes clearly showed the specific features of each reference genome, and that using different kinds of reference map could accelerate fine mapping and map-based cloning of certain genes of cultivated strawberry.

https://www.jstage.jst.go.jp/article/jsbbs/advpub/0/advpub_20151/_pdf

Shiro Nagamatsu

Papers

Classification of strawberry fruit shape by machine learning

Strawberry fruit shape: quantification by image analysis and QTL detection by genome-wide association analysis

Genome-wide Association Study of Strawberry Fruit Quality-related Traits Using a MAGIC Population Derived from Crosses Involving Six Strawberry Cultivars

Detection of Chromosomal Regions for Male Sterility in the Cultivated Strawberry Fragaria × ananassa Duch.

Comparative QTL mapping for male sterility of cultivated strawberry (Fragaria × ananassa Duch.) using different reference genome sequences