All Issue

2024 Vol.33, Issue 4 Preview Page

Original Articles

Melon Fruit Detection and Quality Assessment Using Generative AI-Based Image Data Augmentation

생성형 AI 기반 이미지 데이터 증강을 활용한 멜론 과실 탐지 및 품질 평가

Seungri Yoon1†
,
Yunseong Cho2†
,
Minju Shin3
,
Mi Young Lin3
,
Dahae Kim1
,
Tae In Ahn45*
윤 승리1†
,
조 윤성2†
,
신 민주3
,
임 미영3
,
김 다해1
,
안 태인45*
1Graduate Student, Department of Agriculture, Forestry and Bioresources (Horticultural Science and Biotechnology), Seoul National University, Seoul 08826, Korea
2Graduate Student, Department of Mathematic, Mathematical Data Science, Korea University, Seoul 02841, Korea
3Researcher, Protected Horticulture Researcher Institute, NIHHS, Haman 52054, Korea
4Assistant Professor, Department of Agriculture, Forestry and Bioresources (Horticultural Science and Biotechnology), Seoul National University, Seoul 08826, Korea
5Assistant Professor, Research Institute of Agriculture and Life Sciences, Seoul National University, Seoul 08826, Korea
1서울대학교 농림생물자원학부 대학원생
2고려대학교 수리데이터과학전공 대학원생
3농촌진흥청 국립원예특작과학원 시설원예연구소 연구사
4서울대학교 농림생물자원학부 조교수
5서울대학교 농업생명과학연구원 조교수
*Corresponding Author
†These authors contributed equally to this work.
31 October 2024. pp. 352-360
PDF XML Citation
Abstract

Monitoring and managing the growth and quality of fruits are very important tasks. To train deep learning models like YOLO for real-time fruit detection effectively, high-quality image datasets are essential. However, such datasets are often lacking in agriculture. Generative AI models can help by creating high-quality images. In this study, we used MidJourney and Firefly tools to generate images of melon greenhouses and post-harvest fruits through text-to-image, pre-harvest image-to-image, and post-harvest image-to-image methods. We evaluated these AI-generated images using PSNR and SSIM metrics and tested the YOLOv9 model’s detection performance. We also assessed the net quality of real and generated fruits. Our results showed that generative AI could produce images very similar to real ones, especially for post-harvest fruits. The YOLOv9 model detected the generated images well, and the net quality was also measurable. This shows that generative AI can create realistic images useful for fruit detection and quality assessment, indicating its great potential in agriculture. This study highlights the potential of AI-generated images for data augmentation in melon fruit detection and quality assessment and envisions a positive future for generative AI applications in agriculture.

Keywords
Data augmentation
Deep learning
Fruit detection
Fruit quality
Generative AI
Image-to-image generation
Prompt engineering
Text-to-image generation

현대 농업은 생산성 및 품질 향상과 환경 영향 최소화라는 복합적인 도전에 직면해 있으며, 과실의 성장 및 품질을 모니터링하고 관리하는 것은 매우 중요한 과제이다. YOLO와 같은 실시간 과실 탐지를 위한 딥러닝 모델을 효과적으로 훈련시키기 위해서는 고품질 이미지 데이터셋이 필수적이나, 농업 분야에서는 이러한 데이터셋이 부족한 실정이다. 생성형 AI 모델은 고품질 이미지를 생성하여 이러한 문제를 해결할 수 있다. 본 연구에서는 미드저니와 파이어플라이 도구를 사용하여 텍스트-이미지, 이미지(수확 전)-이미지, 및 이미지(수확 후)-이미지 생성 방법으로 멜론 재배 온실과 수확 후 과실 이미지를 생성하였다. 생성된 이미지는 PSNR 및 SSIM 등의 지표로 평가되었으며, YOLOv9 모델의 탐지 성능(IoU = 0.95)도 평가되었다. 또한 네트 품질 평가 방법을 이용하여 실제 과실과 생성된 과실의 네트 품질을 평가하였다. 생성형 AI는 원본 이미지와 유사한 이미지를 생성하였으며, 수확 후 이미지를 통한 과실 이미지 생성의 경우 매우 뛰어난 사실성을 표현하였다. 생성 이미지는 실제 이미지와 동일하게 YOLOv9 모델에 의해 탐지되었으며, 네트 품질도 평가될 수 있었다. 이는 생성형 AI가 실제적이고 효과적인 과실 탐지 및 품질 평가를 위해 현실적인 이미지를 생성할 수 있음을 보여주며, 농업 분야에서의 큰 잠재력을 나타낸다. 본 연구는 멜론 과실 탐지 및 품질 평가를 위한 데이터 증강에 있어서 AI 생성 이미지의 잠재력을 모색하며, 생성형 AI의 농업에 대한 적용을 긍정적으로 전망한다.

키워드
과실 품질
과실 탐지
데이터 증강
딥러닝
생성형 인공지능
이미지 기반 이미지 생성
프롬프트 엔지니어링
텍스트 기반 이미지 생성
References
1

Abbas A., S. Jain, M. Gour, and S. Vankudothu 2021, Tomato plant disease detection using transfer learning with C-GAN synthetic images. Comput Electron Agric 187:106279.

10.1016/j.compag.2021.106279
2

Akiba Y., A. Ishibashi, M. Sato, and H. Shima 2022, Empirical rule of fruit rind fragmentation in muskmelon netting. J Phys Soc Japan 91:104801.

10.7566/JPSJ.91.104801
3

Bayoudh K., R. Knani, F. Hamdaoui, and A. Mtibaa 2022, A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets. Vis. Comput 38:2939-2970.

10.1007/s00371-021-02166-734131356PMC8192112
4

Bermano A.H., R. Gal, Y. Alaluf, R. Mokady, Y. Nitzan, O. Tov, O. Patashnik, and D. Cohen-Or 2022, State‐of‐the‐Art in the Architecture, Methods and Applications of StyleGAN. Comput Graph Forum 41:591-611. Wiley Online Library.

10.1111/cgf.14503
5

Bird J.J., C.M. Barnes, L.J. Manso, A. Ekárt, and D.R. Faria 2022, Fruit quality and defect image classification with conditional GAN data augmentation. Sci Hortic 293:110684. doi:10.1016/j.scienta.2021.110684

10.1016/j.scienta.2021.110684
6

Chen D., X. Qi, Y. Zheng, Y. Lu, Y. Huang, and Z. Li 2024, Synthetic data augmentation by diffusion probabilistic models to enhance weed recognition. Comput Electron Agric 216:108517.

10.1016/j.compag.2023.108517
7

Chen R.J., M.Y. Lu, T.Y. Chen, D.F.K. Williamson, and F. Mahmood 2021, Synthetic data in machine learning for medicine and healthcare. Nat Biomed Eng 5:493-497.

10.1038/s41551-021-00751-834131324PMC9353344
8

Choi I., S. Park, and J. Park 2022, Generating and modifying high resolution fashion model image using StyleGAN. In 2022 13th International Conference on Information and Communication Technology Convergence (ICTC), pp 1536-1538. IEEE.

10.1109/ICTC55196.2022.9952574
9

Dehouche N., and K. Dehouche 2023, What's in a text-to-image prompt? The potential of stable diffusion in visual arts education. Heliyon 9(6). doi:10.1016/j.heliyon.2023.e16757

10.1016/j.heliyon.2023.e1675737292268PMC10245047
10

Ezura H., and K. Hiwasa-Tanase, 2009. Fruit development. In Plant Developmental Biology-Biotechnological Perspectives: :301-318. Berlin, Heidelberg: Springer Berlin Heidelberg.

10.1007/978-3-642-02301-9_15
11

Gerchikov N., A. Keren-Keiserman, R. Perl-Treves, and I. Ginzberg 2008, Wounding of melon fruits as a model system to study rind netting. Sci Hortic 117:115-122

10.1016/j.scienta.2008.03.015
12

Hodan T., V. Vineet, R. Gal, E. Shalev, J. Hanzelka, T. Connell, P. Urbina, S.N. Sinha, B. Guenter 2019, Photorealistic image synthesis for object instance detection. In 2019 IEEE Int Conf Image Process (ICIP) (pp. 66-70). IEEE.

10.1109/ICIP.2019.8803821
13

Kano Y., and N. Fukuoka 2006, Comparison of cell size and sugar accumulation in melons (Cucumis melo L.) grown early or late in summer. ECB 44:93-102.

10.2525/ecb.44.93
14

Karras T., S. Laine, and T. Aila 2019, A style-based generator architecture for generative adversarial networks. In 2019 IEEE conference on computer vision and pattern recognition (CVPR), pp 4401-4410.

10.1109/CVPR.2019.00453
15

Khalifa N.E., M. Loey, and S. Mirjalili 2022, A comprehensive survey of recent trends in deep learning for digital images augmentation. Artif Intell Rev 55:2351-2377.

10.1007/s10462-021-10066-434511694PMC8418460
16

Leiva-Valenzuela G.A., R. Lu, and J.M. Aguilera 2013, Prediction of firmness and soluble solids content of blueberries using hyperspectral reflectance imaging. J Food Eng 115:91-98.

10.1016/j.jfoodeng.2012.10.001
17

Li L., L. Chang, S. Ke, and D. Huang 2012, Multifractal analysis and lacunarity analysis: A promising method for the automated assessment of muskmelon (Cucumis melo L.) epidermis netting. Comput Electron Agric 88:72-84. doi:10.1016/j.compag.2012.06.006

10.1016/j.compag.2012.06.006
18

Lim M.Y., S.H. Choi, H.J. Jeong, and G.L. Choi 2020, Characteristics of domestic net type melon in hydroponic spring cultivars using coir substrates. Hortic Sci Technol 38:78-86.

10.7235/HORT.20200008
19

Lu C.Y., D.J.A. Rustia, and T.T. Lin 2019, Generative adversarial network based image augmentation for insect pest classification enhancement. IFAC-PapersOnLine 52:1-5.

10.1016/j.ifacol.2019.12.406
20

Lu Y., and S. Young 2020, A survey of public datasets for computer vision tasks in precision agriculture. Comput Electron Agric 178: 105760.

10.1016/j.compag.2020.105760
21

Luo C., Y. Wang, X. Zhang, W. Zhang, and H. Liu 2022, Spatial prediction of soil organic matter content using multiyear synthetic images and partitioning algorithms. Catena 211:106023.

10.1016/j.catena.2022.106023
22

Meor Yahaya M.S., and J. Teo 2023, Data augmentation using generative adversarial networks for images and biomarkers in medicine and neuroscience. Front appl math 9:1162760.

10.3389/fams.2023.1162760
23

Mildenhall B., P. Hedman, R. Martin-Brualla, P.P. Srinivasan, and J.T. Barron 2022, Nerf in the dark: High dynamic range view synthesis from noisy raw images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 16190-16199.

10.1109/CVPR52688.2022.01571
24

Olatunji J.R., G.P. Redding, C.L. Rowe, and A.R. East 2020, Reconstruction of kiwifruit fruit geometry using a CGAN trained on a synthetic dataset. Comput Electron Agric 177:105699.

10.1016/j.compag.2020.105699
25

Sapkota B.B., S. Popescu, N. Rajan, R.G. Leon, C. Reberg-Horton, S. Mirsky, and M.V. Bagavathiannan 2022, Use of synthetic images for training a deep learning model for weed detection and biomass estimation in cotton. Sci Rep 12:19580.

10.1038/s41598-022-23399-z36379963PMC9666527
26

Sapkota R., D. Ahmed, and M. Karkee 2024, Creating image datasets in agricultural environments using DALL.E: generative AI-powered large language model. Soc Sci Res Netw. doi: 10.2139/ssrn.4770726

10.2139/ssrn.4770726
27

Shorten C., and T.M. Khoshgoftaar 2019, A survey on image data augmentation for deep learning. J Bio Data 6:1-48.

10.1186/s40537-019-0197-0
28

Vo H.T., K.C. Mui, N.N. Thien, and P.P. Tien 2024, Automating Tomato Ripeness Classification and Counting with YOLOv9. Int J Adv Comput Sci Appl 15.

10.14569/IJACSA.2024.01504113
29

Wang D., J.G. Wang, and K. Xu 2021, Deep learning for object detection, classification and tracking in industry applications. Sensors 21:7349.

10.3390/s2121734934770656PMC8587754
30

Wang Z., A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli 2004, Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13:600-612.

10.1109/TIP.2003.81986115376593
31

Wong S.C., A. Gatt, V. Stamatescu, and M.D. McDonnell 2016, Understanding data augmentation for classification: when to warp?. In 2016 International Conference on Digital Image Computing: Techniques and Applications, 1-6.

10.1109/DICTA.2016.7797091
32

Yoon S., M. Shin, J.H. Kim, J.W. Bang, H.J. Jeong, and T.I. Ahn 2023, Analysis of the relationship between melon fruit growth and net quality using computer vision and fruit modeling. J Bio-Env Con 32(4), 456-465.

10.12791/KSBEC.2023.32.4.456
33

Yu F., A. Seff, Y. Zhang, S. Song, T. Funkhouser, and J. Xiao 2015, Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365.

34

Zhao Z.Q., P. Zheng, S.T. Xu, and X. Wu 2019, Object detection with deep learning: A review. IEEE Trans Neural Netw Learn Syst 30:3212-3232.

10.1109/TNNLS.2018.287686530703038
35

Zhou D., J. Fang, X. Song, C. Guan, J. Yin, Y. Dai, and R. Yang 2019, Iou loss for 2d/3d object detection. In 2019 International Conference on 3D Vision (3DV), pp 85-94. IEEE.

10.1109/3DV.2019.00019
Information
  • Publisher :The Korean Society for Bio-Environment Control
  • Publisher(Ko) :(사)한국생물환경조절학회
  • Journal Title :Journal of Bio-Environment Control
  • Journal Title(Ko) :생물환경조절학회지
  • Volume : 33
  • No :4
  • Pages :352-360
  • Received Date : 2024-07-06
  • Revised Date : 2024-10-08
  • Accepted Date : 2024-10-21
  • DOI :https://doi.org/10.12791/KSBEC.2024.33.4.352
Journal Informaiton Journal of Bio-Environment Control Journal of Bio-Environment Control
Online Submission
  • NRF
  • KOFST
  • crossref crossmark
  • crossref cited-by
  • crossref funder-registry
  • crosscheck
  • orcid
  • open access
  • ccl
Journal Informaiton Journal Informaiton - close