Counting Sweet Corn Plants and Estimating Leaf Area and Leaf Color from Images

Manually counting sweet corn plants and measuring leaf area in the field is tedious. I tried automating this with object detection and segmentation models, and found that a general-purpose YOLO pretrained on COCO simply had no "corn" class to detect, while SAM3, an open-vocabulary segmentation model, could count plants and estimate leaf area and leaf color at a practical level using just the text prompt "crop." That said, overlapping plants still caused over- and under-counting, and merged multiple plants into a single detection.

Background: Why Estimate Plant Count and Leaf Area from Images

Sweet corn growth surveys usually involve walking the field, counting plants by eye, and measuring leaf size visually or with a ruler. This is accurate, but it doesn't scale — the larger the field and the more survey points and growth stages you want to track, the more the survey cost balloons.

If plant count and leaf area (a proxy for how bushy the foliage is) could be estimated from images, growth could be tracked quantitatively using nothing more than video or photos taken with a smartphone or drone. If leaf color (the green-to-yellow tendency) could also be captured at the same time, it could serve as a simple indicator of growth stage or nitrogen status. That was the starting point for this experiment.

First Attempt: YOLO and the COCO Class Wall

The first object detection model I tried was the well-known YOLO (You Only Look Once). YOLO is extremely fast at inference and is widely used for real-time detection, but pretrained models are trained on the COCO dataset (80 classes covering people, vehicles, animals, and so on).

Naturally, COCO's 80 classes don't include anything like "corn" or "crop." So using a pretrained YOLO model as-is, there was no way to detect corn plants. Detecting a custom class would require fine-tuning on your own annotated data, but since the goal here was first to see how far an off-the-shelf model could get, I gave up on YOLO at this point.

A Taxonomy of Object Detection Models: Why I Turned to SAM3

After hitting a wall with YOLO, I re-organized object detection and segmentation models along two axes.

Axis 1: Classification by Detectable Vocabulary

• Closed-set detection: Models that can only detect a fixed set of classes defined at training time. General-purpose COCO-trained models like YOLOv8 fall here. Inference is fast, but unknown classes (in this case, "corn plants") simply cannot be detected.
• Open-vocabulary / zero-shot detection: Models that let you specify what to detect via a text prompt at inference time. They attempt to detect classes never seen during training, at least to some degree. SAM3, YOLO-World, and Grounding DINO — all three models tried here — fall into this category.

Axis 2: Classification by Output Format

• Bounding-box detection: Models that simply enclose a target in a rectangle (bbox). YOLO, YOLO-World, and Grounding DINO fall here. Because the output is a rectangle, it tends to pull in background and adjacent plants, making it poorly suited to accurate area measurement.
• Segmentation: Models that extract a target as a pixel-level mask. The SAM (Segment Anything Model) family falls here. Because you get a mask, you can compute the area and color of the vegetation directly, with the background excluded.

Given that this task needed to detect an unknown class (corn) while also getting pixel-level leaf area information, the combination of "open-vocabulary × segmentation" made the most sense. SAM3 is the model that satisfies both conditions.

Detection with SAM3: Practical-Level Accuracy

SAM3 (Segment Anything Model 3) isn't trained on a fixed class set like COCO — it can segment arbitrary concepts given a text prompt. I used short English prompts like corn plant and crop, and tested on both still images and video.

The videos were shot in portrait orientation at 720×1280, roughly 30 fps, but running inference at the full frame rate is more compute than necessary, so I downsampled to around 10 fps. That said, downsampling too aggressively noticeably degrades tracking performance, so this needs to be tuned carefully. I ran everything on my own RTX 5090, and found that around 80 frames was roughly the limit before a single run risked running out of memory, so I split each video into chunks of that size and processed them separately.

Here is the result.

SAM3's Challenge: Over- and Under-Counting from Overlapping Plants

That said, using SAM3's raw detection output as-is still left some challenges. Two patterns stood out in particular:

• Fluctuating counts where plants overlap: Where neighboring plants had entangled leaves, a single plant would sometimes be over-detected as two, or conversely two plants would be under-detected as one clump.
• Multiple plants merged into a single detection: Especially in later growth stages with dense foliage, the leaves of two or three adjacent plants were sometimes segmented together as a single unit.

To address this, I added post-processing that looks for connected components (physically separate blobs within a mask) and re-splits the mask when one component is large relative to the total area. However, when leaves are so overlapped that their outlines are actually continuous, connected-component analysis can't separate them, and in those cases visual confirmation by a human is still necessary.

The number of cases automatically flagged as likely duplicates was small relative to the total, but this only catches clear-cut duplicates based on similarity in area, color, and mask shape — it's worth noting that ambiguous boundary cases may still slip through uncaught.

Comparison: YOLO-World and Grounding DINO

Besides SAM3, I also tried YOLO-World and Grounding DINO, both open-vocabulary bounding-box detection models capable of detecting classes beyond COCO. Both share SAM3's ability to specify an arbitrary class via text prompt, but when I actually pointed them at corn plants, neither matched SAM3's accuracy.

Here is the YOLO World result.

Here is the Grounding DINO result.

A likely reason is that YOLO-World and Grounding DINO are fundamentally bounding-box detectors and don't produce pixel-level masks the way SAM3 does, so bounding boxes overlap heavily wherever the boundary between adjacent plants is ambiguous. It's also possible that SAM3, trained on the large and diverse segmentation data behind the Segment Anything line of models, simply generalizes better to irregular, natural shapes like crops than bounding-box-based models do.

Either way, for a task that required both counting plants and getting leaf area/color, the segmentation-based SAM3 produced results closest to practical use.

Conclusion

A general-purpose COCO-trained YOLO couldn't detect a class it had never seen (corn plants), and was unusable for this task. SAM3, an open-vocabulary segmentation model, on the other hand, was able to handle plant detection, tracking, and leaf area/color estimation at a practical level using nothing more than the text prompt "crop."

That said, challenges remain around over- and under-counting where plants overlap, and around multiple plants being merged into a single detection — fully automating plant counts in densely planted fields still needs more work. I also tried other open-vocabulary detectors, YOLO-World and Grounding DINO, but being bounding-box based, neither matched SAM3's accuracy. Next steps include strengthening the overlap-handling logic and tuning prompts for different growth stages.

FAQ

Q: Why couldn't YOLO detect corn plants?
A: General pretrained YOLO models are trained on the COCO dataset (80 classes covering people, vehicles, animals, and so on), which doesn't include a class corresponding to "corn" or "crop." Detecting a custom class requires fine-tuning on your own data.

Q: What's the difference between SAM3 and YOLO?
A: YOLO is a closed-set, bounding-box detection model — it can only detect classes it was trained on. SAM3 is an open-vocabulary segmentation model that lets you specify an arbitrary target via text prompt, and it outputs pixel-level masks, which makes it easier to detect unseen targets and lets you compute area directly.

Q: What should I watch out for when counting plants with SAM3?
A: Where plants overlap, you can get over- or under-counting, and multiple plants can be segmented as a single unit. Post-processing that splits masks by connected components can help, but when leaves are fully overlapping it's hard to separate them automatically, so combining this with visual confirmation is recommended.

Q: Can YOLO-World or Grounding DINO be used instead of SAM3?
A: Both support open-vocabulary detection, but since they only output bounding boxes and no pixel-level mask, they fell short of SAM3 in detection accuracy for ambiguous plant boundaries and in computing leaf area.