All Issue

2026 Vol.35, Issue 1 Preview Page

Original Articles

Development of a Text-Guided Annotation System for Object Segmentation Training Data in Crop Images Based on the Segmentation Anything Model 3

작물 이미지 객체 분할 학습데이터 구축을 위한 Segmentation Anything Model 3 기반 텍스트 유도형 어노테이션 시스템 개발

Seungwon Seok1
,
Hye-Sung Park2
,
Solhee Kim3
,
Youngmi Lee45
,
Taegon Kim67*
석 승원1
,
박 혜성2
,
김 솔희3
,
이 영미45
,
김 태곤67*
1Master’s Student, Department of Agricultural Engineering, College of Agriculture & Life Sciences, Jeonbuk National University, Jeonju 54896, Korea
2Researcher, Mushroom Research Division, National Institute of Horticultural and Herbal Science, Eumseong 27790, Korea
3Associate Research Professor, Department of Smart Farm, College of Agriculture & Life Sciences, Jeonbuk National University, Jeonju 54896, Korea
4Assistant Professor, Department of Statistics, College of Natural Science, Jeonbuk National University, Jeonju 54896, Korea
5Research Institute of Applied Statistics, Jeonbuk National University, Jeonju 54896, Korea
6Associate Professor, Department of Smart Farm, College of Agriculture & Life Sciences, Jeonbuk National University, Jeonju 54896, Korea
7Researcher, Institute of Agricultural Science & Technology, Jeonbuk National University, Jeonju 54896, Korea
1전북대학교 농업생명과학대학 농공학과 석사과정
2국립원예특작과학원 인삼특작부 버섯과 연구사
3전북대학교 농업생명과학대학 스마트팜학과 연구부교수
4전북대학교 자연과학대학 통계학과 조교수
5전북대학교 응용통계연구소 연구원
6전북대학교 농업생명과학대학 스마트팜학과 부교수
7전북대학교 농업과학기술연구소 연구원
*Corresponding Author
31 January 2026. pp. 26-39
PDF XML Citation
Abstract

With the widespread adoption of computer vision technology in agriculture, securing high-quality training data has become essential. However, existing data construction methods are limited by the substantial time and cost required. In this work, we develop a semi-automatic annotation system that integrates SAM3 (Segment Anything Model 3), a state-of-the-art multimodal foundation model. SAM3(Segment Anything Model 3). The proposed system is implemented in a GUI environment and consists of three stages: (1) object recognition based on text prompts, (2) generation of precise masks and their conversion into trainable polygon coordinates, and (3) quality assurance through user verification. In performance evaluations using 600 images, SAM3 achieved a matching rate of 92.9% and a mean Average Precision (mAP) of 0.790, reducing dataset construction time by 96－98% compared to manual annotation. These results demonstrate superior performance in both accuracy and efficiency compared to baseline foundation models such as SAM+CLIP and Grounding DINO+SAM. This study highlights how the zero-shot capabilities of foundation models can drastically improve agricultural data labeling efficiency and accelerate related AI research.

Keywords
agricultural dataset
computer vision
deep learning
foundation model
image annotation

농업 분야 컴퓨터 비전(Computer Vision) 기술 확산으로 고품질 학습 데이터 확보가 필수적이나, 기존의 수동 데이터 구축 방식은 많은 시간과 비용이 소요되는 한계가 있다. 이에 본 연구는 최신 멀티모달 파운데이션 모델인 SAM3(Segment Anything Model 3)를 기반으로 반자동 어노테이션 시스템을 개발하였다. 제안 시스템은 (1) 텍스트 프롬프트 기반 객체 인식, (2) SAM3 기반 정밀 마스크 생성 및 학습 가능한 폴리곤 좌표 변환, (3) 사용자 검증의 3단계로 구성되며 GUI로 구현되었다. 600장 이미지 평가 결과, SAM3는 92.9%의 매칭률과 0.790의 평균 정밀도(mAP)를 달성하였으며, 데이터셋 구축 시간을 수동 작업 대비 96~98% 단축시켰다. 이는 SAM+CLIP, Grounding DINO+SAM 등 기존 파운데이션 모델 대비 정확도와 효율성 모든 면에서 월등한 성능이다. 본 연구는 파운데이션 모델의 제로샷 성능을 활용해 농업 데이터 레이블링 효율을 개선하고 관련 AI 연구 가속화에 기여할 것으로 기대된다.

키워드
농업 데이터셋
딥러닝
이미지 어노테이션
컴퓨터 비전
파운데이션 모델
References
1

Ahmed R, Nova TT, Jarin TF, Hossain MM, Shams K, Rashid MRA (2025) Real-time monitoring of oyster mushroom cultivation using CCTV and attention-enhanced ShuffleNet-based explainable AI techniques. Smart Agric Technol 101571. https://doi.org/10.1016/j.atech.2025.101571

10.1016/j.atech.2025.101571
2

Carion N, Gustafson L, Hu YT, Debnath S, Hu R, Suris D, Ryali C, Alwala KV, Khedr H, Huang A, Lei J, Ma T, Guo B, Kalla A, Marks M, Greer J, Wang M, Sun P, Rädle R, Afouras T, Mavroudi E, et al. (2025) SAM 3: Segment Anything with Concepts. arXiv preprint arXiv:2511.16719. https://doi.org/10.48550/arXiv.2511.16719

10.48550/arXiv.2511.16719
3

Charisis C, Nuwayhid S, Argyropoulos D (2025) A novel Mask R-CNN-based tracking pipeline for oyster mushroom cluster growth monitoring in time-lapse image datasets. Comput Electron Agric 237:110590. https://doi.org/10.1016/j.compag.2025.110590

10.1016/j.compag.2025.110590
4

Choi YW, Kim NE, Paudel B, Kim HT (2022) Strawberry pests and diseases detection technique optimized for symptoms using deep learning algorithm. J Bio-Env Con 31:255-263. (in Korean) https://doi.org/10.12791/KSBEC.2022.31.3.255

10.12791/KSBEC.2022.31.3.255
5

Gao P, Geng S, Zhang R, Ma T, Fang R, Zhang Y, Li H, Qiao Y (2024) CLIP-Adapter: better vision-language models with feature adapters. Int J Comput Vis 132:581-595. https://doi.org/10.1007/s11263-023-01891-x

10.1007/s11263-023-01891-x
6

Ghazal S, Munir A, Qureshi WS (2024) Computer vision in smart agriculture and precision farming: Techniques and applications. Artif Intell Agric 13:64-83. https://doi.org/10.1016/j.aiia.2024.06.004

10.1016/j.aiia.2024.06.004
7

Guragain DP, Shrestha B, Bajracharya I, Sharma N, Ghimire D, Bhat SA (2025) OysterMushNet: A deep learning-based mobile application for real-time mushroom disease detection and agronomist support. Smart Agric Technol 101735. https://doi.org/10.1016/j.atech.2025.101735

10.1016/j.atech.2025.101735
8

He H (2023) Segment_anything_annotator. GitHub repository. https://github.com/haochenheheda/segment-anything-annotator

9

He K, Gkioxari G, Dollár P, Girshick R (2017) Mask R-CNN. Proc IEEE Int Conf Comput Vis 2961-2969. https://doi.org/10.1109/ICCV.2017.322

10.1109/ICCV.2017.322
10

Ji W, Li J, Bi Q, Liu T, Li W, Cheng L (2024) Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applications. Mach Intell Res 21:1071-1090. https://doi.org/10.1007/s11633-023-1385-0

10.1007/s11633-023-1385-0
11

Kamilaris A, Prenafeta-Boldú FX (2018) Deep learning in agriculture: A survey. Comput Electron Agric 147:70-90. https://doi.org/10.1016/j.compag.2018.02.016

10.1016/j.compag.2018.02.016
12

Kirillov A, Mintun E, Ravi N, Mao H, Rolland C, Gustafson L, Xiao T, Whitehead S, Berg AC, Lo WY, Dollár P, Girshick R (2023) Segment Anything. Proc IEEE/CVF Int Conf Comput Vis 4015-4026. https://doi.org/10.1109/ICCV51070.2023.00371

10.1109/ICCV51070.2023.00371
13

Koirala A, Walsh KB, Wang Z, McCarthy C (2019) Deep learning - Method overview and review of use for fruit detection and yield estimation. Comput Electron Agric 162:219-234. https://doi.org/10.1016/j.compag.2019.04.017

10.1016/j.compag.2019.04.017
14

Korea Institute of Science and Technology Information (KISTI) (2019) Development of optimal growth model for precision cultivation of oyster mushroom and a standardization of oyster mushroom house. National Research Report. TRKO201900015947.

15

Lee CJ, Park HS, Lee EJ, Kong WS, Yu BK (2019) Analysis of growth environment by smart farm cultivation of oyster mushroom ‘Chunchu No 2’. J Mushrooms 17:119-125. https://doi.org/10.14480/JM.2019.17.3.119

10.14480/JM.2019.17.3.119
16

Lee SH, Yu BK, Kim HJ, Yun NK, Jung JC (2015) Technology for improving the uniformity of the environment in the oyster mushroom cultivation house by using multi-layered shelves. J Bio-Env Con 24:128-133. (in Korean) http://dx.doi.org/10.12791/KSBEC.2015.24.2.128

10.12791/KSBEC.2015.24.2.128
17

Li B, Weinberger KQ, Belongie S, Koltun V, Ranftl R (2022) Language-driven Semantic Segmentation. arXiv preprint arXiv:2201.03546. https://doi.org/10.48550/arXiv.2201.03546

10.48550/arXiv.2201.03546
18

Li Y, Wang D, Yuan C, Li H, Hu J (2023) Enhancing Agricultural Image Segmentation with an Agricultural Segment Anything Model Adapter. Sensors 23:7884. https://doi.org/10.3390/s23187884

10.3390/s2318788437765940PMC10534855
19

Li Z, Guo R, Li M, Chen Y, Li G (2020) A review of computer vision technologies for plant phenotyping. Comput Electron Agric 176:105672. https://doi.org/10.1016/j.compag.2020.105672

10.1016/j.compag.2020.105672
20

Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollar P, Zitnick CL (2014) Microsoft COCO: Common objects in context. Proc. Eur. Conf. Comput. Vis. pp 740-755. https://doi.org/10.1007/978-3-319-10602-1_48

10.1007/978-3-319-10602-1_48
21

Liu S, Zeng Z, Ren T, Li F, Zhang H, Yang J, Zhang L (2024) Grounding DINO: Marrying DINO with Grounded Pre-training for Open-Set Object Detection. Eur Conf Comput Vis 38-55. https://doi.org/10.1007/978-3-031-72970-6_3

10.1007/978-3-031-72970-6_3
22

Ma X, Wu Q, Zhao X, Zhang X, Pun MO, Huang B (2024) SAM-Assisted remote sensing imagery semantic segmentation with object and boundary constraints. IEEE Trans Geosci Remote Sens 62:1-16. https://doi.org/10.1109/TGRS.2024.3443420

10.1109/TGRS.2024.3443420
23

Mazurowski MA, Dong H, Gu H, Yang J, Konz N, Zhang Y (2023) Segment anything model for medical image analysis: An experimental study. Med Image Anal 89:102918. https://doi.org/10.1016/j.media.2023.102918

10.1016/j.media.2023.10291837595404PMC10528428
24

Mumuni F, Mumuni A (2024) Segment Anything Model for automated image data annotation: empirical studies using text prompts from Grounding DINO. arXiv preprint arXiv:2406.19057. https://doi.org/10.48550/arXiv.2406.19057

10.48550/arXiv.2406.19057
25

Radford A, Kim JW, Hallacy C, Ramesh A, Goh G, Agarwal S, Sutskever I (2021) Learning Transferable Visual Models From Natural Language Supervision. Int Conf Mach Learn 8748-8763. https://doi.org/10.48550/arXiv.2103.00020

10.48550/arXiv.2103.00020
26

Rao Y, Zhao W, Chen G, Tang Y, Zhu Z, Huang G, Lu J (2022) DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting. Proc IEEE/CVF Conf Comput Vis Pattern Recognit 18082-18091. https://doi.org/10.1109/CVPR52688.2022.01755

10.1109/CVPR52688.2022.01755
27

Redmon J, Divvala S, Girshick R, Farhadi A (2016) You Only Look Once: Unified, Real-Time Object Detection. Proc IEEE Conf Comput Vis Pattern Recognit 779-788. https://doi.org/10.1109/CVPR.2016.91

10.1109/CVPR.2016.91
28

Royse DJ, Baars J, Tan Q (2017) Current overview of mushroom production in the world. In: Zied DC, Pardo-Giménez A (eds) Edible and medicinal mushrooms: Technology and applications. Wiley, West Sussex, UK, pp 5-13. https://doi.org/10.1002/9781119149446.ch2

10.1002/9781119149446.ch2
29

Russell BC, Torralba A, Murphy KP, Freeman WT. 2008. LabelMe: a database and web-based tool for image annotation. Int J Comput Vis 77:157-173. https://doi.org/10.1007/s11263-007-0090-8

10.1007/s11263-007-0090-8
30

Sánchez C (2010) Cultivation of Pleurotus ostreatus and other edible mushrooms. Appl Microbiol Biotechnol 85:1321-1337. https://doi.org/10.1007/s00253-009-2343-7

10.1007/s00253-009-2343-7
31

Silva C, Costa D, Costa J, Ribeiro B (2024) Data annotation quality in smart farming industry. Prod Manuf Res 12:2377253. https://doi.org/10.1080/21693277.2024.2377253

10.1080/21693277.2024.2377253
32

Singh R, Bidese R, Dhakal K, Sornapudi S (2025) Few-Shot Adaptation of Grounding DINO for Agricultural Domain. Proc Comput Vis Pattern Recognit Conf 5332-5342. https://doi.org/10.1109/CVPRW67362.2025.00530

10.1109/CVPRW67362.2025.00530
33

Taupa GJ, Villarica MV, Vinluan AA (2024) Enhancing oyster mushroom cultivation with solar-powered IoT and machine learning: Predicting harvest readiness. J Inf Syst Eng Manag 10:5551. https://doi.org/10.52783/jisem.v10i33s.5551

10.52783/jisem.v10i33s.5551
34

Wosner O, Farjon G, Bar-Hillel A (2021) Object detection in agricultural contexts: A multiple resolution benchmark and comparison to human. Comput Electron Agric 189:106404. https://doi.org/10.1016/j.compag.2021.106404

10.1016/j.compag.2021.106404
35

Yin S, Xi Y, Zhang X, Sun C, Mao Q (2025). Foundation Models in Agriculture: A Comprehensive Review. Agriculture 15:847. https://doi.org/10.3390/agriculture15080847

10.3390/agriculture15080847
36

Yoon S, Shin M, Kim JH, Jeong HJ, Park J, Ahn TI (2024) Computer Vision Approach for Phenotypic Characterization of Horticultural Crops. J Bio-Env Con 33:63-70. https://doi.org/10.12791/KSBEC.2024.33.1.063

10.12791/KSBEC.2024.33.1.063
37

Zhong Y, Yang J, Zhang P, Li C, Codella N, Li LH, Zhou L, Dai X, Yuan L, Li Y, Gao J (2022) RegionCLIP: Region-based language-image pretraining. Proc IEEE/CVF Conf Comput Vis Pattern Recognit, 16793-16803. https://doi.org/10.1109/CVPR52688.2022.01629

10.1109/CVPR52688.2022.01629
Information
  • Publisher :The Korean Society for Bio-Environment Control
  • Publisher(Ko) :(사)한국생물환경조절학회
  • Journal Title :Journal of Bio-Environment Control
  • Journal Title(Ko) :생물환경조절학회지
  • Volume : 35
  • No :1
  • Pages :26-39
  • Received Date : 2026-01-04
  • Revised Date : 2026-01-14
  • Accepted Date : 2026-01-22
  • DOI :https://doi.org/10.12791/KSBEC.2026.35.1.026
Journal Informaiton Journal of Bio-Environment Control Journal of Bio-Environment Control
Online Submission
  • NRF
  • KOFST
  • crossref crossmark
  • crossref cited-by
  • crossref funder-registry
  • crosscheck
  • orcid
  • open access
  • ccl
Journal Informaiton Journal Informaiton - close