In the Kantor Lab at CMU, surveying fields to sample crop phenotypes for breeding or farm management was a task of interest. In this particular project, we tested the efficacy of reconstructing the sorghum stalk from a series of arm-mounted images, and then did our best to estimate seed count and seed weight from the reconstructions. Generally speaking we ran 2D seed detection algorithms on individual images, then adjusted the 3D points into alignment via ICP on meaningful local features, then used 3D location to combine the series of 2D seed detections into a combined 3D count. For more information you can check out the paper on IEEE or arXiv. This paper was accepted at ICRA 2023.

I think we developed a somewhat interesting method of measuring reconstruction quality when there was no ground truth as to the camera poses or seed positions. We would render a view of the point cloud and then compare that rendered snippet with a snippet from the original images. Through a series of trials with synthetic noise we found that certain image processing steps had a reasonably monotonic relationship with noise. For example, the difference between the Laplacian of a normalized greyscale image for both the original snippet and rendered snippet grew as the synthetic noise increased. Therefore we used this operation to measure the deterioration in reconstruction quality during process ablations. More difference in the Laplacian between the original and render indicated a worse reconstruction job.