All Issue

2022 Vol.31, Issue 4 Preview Page

Original Articles

Grading of Harvested ‘Mihwang’ Peach Maturity with Convolutional Neural Network

합성곱 신경망을 이용한 ‘미황’ 복숭아 과실의 성숙도 분류

Mi Hee Shin1†
,
Kyeong Eun Jang2†
,
Seul Ki Lee3
,
Jung Gun Cho3
,
Sang Jun Song4
,
Jin Gook Kim56*
신 미희1†
,
장 경은2†
,
이 슬기3
,
조 정건3
,
송 상준4
,
김 진국56*
1Junior Researcher, Institute of Agriculture and Life Sciences, Gyeongsang National University, Jinju, 52828, Korea
2Graduate Student, Division of Applied Life Science, Graduate School of Gyeongsang National University, Jinju 52828, Korea
3Researcher, Fruit Research Division, National Institute of Horticultural and Herbal Science, Wanju 55365, Korea
4Representative, Farm&Farm Soft, Hanam 12912, Korea
5Professor, Department of Horticulture, College of Agriculture and Life Science, Gyeongsang National University, Jinju 52828, Korea
6Principal Researcher, Institute of Agriculture and Life Sciences, Gyeongsang National University, Jinju 52828, Korea
1경상국립대학교 농업생명과학연구원 선임연구원
2경상국립대학교 대학원 응용생명과학부 대학원생
3국립원예특작과학원 과수과 농업연구사
4팜앤팜소프트 대표
5경상국립대학교 원예학과 교수
6경상국립대학교 농업생명과학연구원 책임연구원
*Corresponding Author
†These authors contributed equally to this work.
31 October 2022. pp. 270-278
PDF XML Citation
Abstract

This study was conducted using deep learning technology to classify for ‘Mihwang’ peach maturity with RGB images and fruit quality attributes during fruit development and maturation periods. The 730 images of peach were used in the training data set and validation data set at a ratio of 8:2. The remains of 170 images were used to test the deep learning models. In this study, among the fruit quality attributes, firmness, Hue value, and a* value were adapted to the index with maturity classification, such as immature, mature, and over mature fruit. This study used the CNN (Convolutional Neural Networks) models for image classification; VGG16 and InceptionV3 of GoogLeNet. The performance results show 87.1% and 83.6% with Hue left value in VGG16 and InceptionV3, respectively. In contrast, the performance results show 72.2% and 76.9% with firmness in VGG16 and InceptionV3, respectively. The loss rate shows 54.3% and 62.1% with firmness in VGG16 and InceptionV3, respectively. It considers increasing for adapting a field utilization with firmness index in peach.

Keywords
convolutional neural networks
deep learning
GoogLeNet
maturity
robot harvest

본 연구는 무대재배 복숭아 ‘미황’을 대상으로 성숙기간 중 RGB 영상을 취득한 후 다양한 품질 지표를 측정하고 이를 딥러닝 기술에 적용하여 복숭아 과실 숙도 분류의 가능성을 탐색하고자 실시하였다. 취득 영상 730개의 데이터를 training과 validation에 사용하였고, 170개는 최종 테스트 이미지로 사용하였다. 본 연구에서는 딥러닝을 활용한 성숙도 자동 분류를 위하여 조사된 품질 지표 중 경도, Hue 값, a*값을 최종 선발하여 이미지를 수동으로 미성숙(immature), 성숙(mature), 과숙(over mature)으로 분류하였다. 이미지 자동 분류는 CNN (Convolutional Neural Networks, 컨볼루션 신경망) 모델 중에서 이미지 분류 및 탐지에서 우수한 성능을 보이고 있는 VGG16, GoogLeNet의 InceptionV3 두 종류의 모델을 사용하여 복숭아 품질 지표 값의 분류 이미지별 성능을 측정하였다. 딥러닝을 통한 성숙도 이미지 분석 결과, VGG16과 InceptionV3 모델에서 Hue_left 특성이 각각 87.1%, 83.6%의 성능(F1 기준)을 나타냈고, 그에 비해 Firmness 특성이 각각 72.2%, 76.9%를 나타냈고, Loss율이 각각 54.3%, 62.1%로 Firmness를 기준으로 한 성숙도 분류는 적용성이 낮음을 확인하였다. 추후에 더 많은 종류의 이미지와 다양한 품질 지표를 가지고 학습이 진행된다면 이전 연구보다 향상된 정확도와 세밀한 성숙도 판별이 가능할 것으로 판단되었다.

키워드
구글넷
딥러닝
로봇수확
숙도
합성곱 신경망
References
1
Bang J.H., J. Park, S.W. Park, J.Y. Kim, S.H. Jung, and C.B. Sim 2022, A system for determining the growth stage of fruit tree using a deep learning-based object detection model. Smart Media J 11:9-18. (in Korean) doi:10.30693/SMJ.2022.11.4.9 10.30693/SMJ.2022.11.4.9
2
Behera S.K., A.K. Rath, and P.K. Sethy 2021, Maturity status classification of papaya fruits based on machine learning and transfer learning approach. Inf Process Agric 8:244-250. doi:10.1016/j.inpa.2020.05.003 10.1016/j.inpa.2020.05.003
3
Bhole V., and A. Kumar 2020, Mango quality grading using deep learning technique: Perspectives from agriculture and food industry. Proceedings of the 21st Annual Conference on Information Technology Education, pp 180-186. doi:10.1145/3368308.3415370 10.1145/3368308.3415370
4
Chun J.P., and J.G. Kim 2013, Changes of quality indices and cell wall components during ripening and softening in peach fruit (Prunus persica L. Batsch). J Agric Life Sci 47:35-42. (in Korean)
5
Gong L., W. Wang, T. Wang, and C. Liu 2022, Robotic harvesting of the occluded fruits with a precise shape and position reconstruction approach. J Field Robotics 39:69-84. doi:10.1002/rob.22041 10.1002/rob.22041
6
Ham H.S., and H.C. Cho 2020, A development of rose leaf disease classification system using convolutional neural network. Trans Korean Inst Electr Eng 69:1040-1045. (in Korean) doi:10.5370/KIEE.2020.69.7.1040 10.5370/KIEE.2020.69.7.1040
7
Jeong S.H., M.H. Lee, and H. Yoe 2018, Fruit classification system using deep learning. J Knowl Inf Technol Syst 13:589-595. (in Korean) doi:10.34163/jkits.2018.13.5.009 10.34163/jkits.2018.13.5.009
8
Kim J.K., W. Cho, M.H. Na, and M. Chun 2019, Development of automatic classification system of vegetables by image processing and deep learning. J Korean Data Anal Soc 21:63-73. (in Korean) doi:10.37727/jkdas.2019.21.1.63 10.37727/jkdas.2019.21.1.63
9
Kim M., and C. Choi 2020, Superpixel-based apple leaf disease classification using convolutional neural network. J Broadcast Eng 25:208-217. (in Korean) doi:10.5909/JBE.2020.25.2.208 10.5909/JBE.2020.25.2.208
10
Kingma D.P., and J. Ba 2015, Adam: A method for stochastic optimization. 3rd International Conference on Learning Representations. doi:10.48550/arXiv.1412.6980 10.48550/arXiv.1412.6980
11
LeCun Y., B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, and L.D. Jackel 1989, Backpropagation applied to handwritten zip code recognition. Neural Comput 1:541-551. doi:10.1162/neco.1989.1.4.541 10.1162/neco.1989.1.4.541
12
Li X., C. Cai, H. Zheng, and H. Zhu 2022, Recognizing strawberry appearance quality using different combinations of deep feature and classifiers. J Food Process Eng 45:e13982. doi:10.1111/jfpe.13982 10.1111/jfpe.13982
13
Lim S.C., S.H. Kim, Y.H. Kim, and D.Y. Kim 2017, Training network design based on convolution neural network for object classification in few class problem. J Korea Inst Inf Commun Eng 21:144-150. (in Korean) doi:10.6109/jkiice.2017.21.1.144 10.6109/jkiice.2017.21.1.144
14
Mazen F.M.A., and A.A. Nashat 2019, Ripeness classification of bananas using an artificial neural network. Arab J Sci Eng 44:6901-6910. doi:10.1007/s13369-018-03695-5 10.1007/s13369-018-03695-5
15
Moon T., J. Park, and J.E. Son 2020, Estimation of sweet pepper crop fresh weight with convolutional neural network. Protected Hort Plant Fac 29:381-387. (in Korean) doi:10.12791/KSBEC.2020.29.4.381 10.12791/KSBEC.2020.29.4.381
16
Park J.Y., H.J. Kim, and K. Kim 2020, Accessing impact of DCGAN image data augmentation for CNN based tomato disease classification. J Digital Cont Soc 21:959-967. (in Korean) doi:10.9728/dcs.2020.21.5.959 10.9728/dcs.2020.21.5.959
17
Saleem M.H., J. Potgieter, and K.M. Arif 2021, Automation in agriculture by machine and deep learning techniques: A review of recent developments. Precis Agric 22:2053-2091 doi:10.1007/s11119-021-09806-x 10.1007/s11119-021-09806-x
18
Seo D., K.C. Kim, M. Lee, K.D. Kwon, and G. Kim 2021, Research on tomato flowers and fruits object detection model in greenhouse environment using deep learning. J Korean Inst Commun Inf Sci 46:2072-2077. (in Korean) doi:10.7840/kics.2021.46.11.2072 10.7840/kics.2021.46.11.2072
19
Silalahi D.D., C.E. Reaño, F.P. Lansigan, R.G. Panopio, and N.C. Bantayan 2016, Using genetic algorithm neural network on near infrared spectral data for ripeness grading of oil palm fresh fruit. Inf Process Agric 3:252-261. doi:10.1016/j.inpa.2016.10.001 10.1016/j.inpa.2016.10.001
20
Stasenko N., E. Chernova, D. Shadrin, G. Ovchinnikov, L. Krivolapov, and M. Pukalchik 2021, Deep learning for improving the storage process: Accurate and automatic segmentation of spoiled areas on apples. 2021 IEEE Instrumentation and Measurement Technology Conference 1-6. doi:10.1109/I2MTC50364.2021.9460071 10.1109/I2MTC50364.2021.9460071
21
Tan K., W.S. Lee, H. Gan, and S. Wang 2018, Recognising blueberry fruit of different maturity using histogram oriented gradients and colour features in outdoor scenes. Biosyst Eng 176:59-72. doi:10.1016/j.biosystemseng.2018.08.011 10.1016/j.biosystemseng.2018.08.011
22
Wang D., W. Cao, F. Zhang, Z. Li, S. Xu, and X. Wu 2022, A review of deep learning in multiscale agricultural sensing. Remote Sens 14:559. doi:10.3390/rs14030559 10.3390/rs14030559
23
Xu M., J. Sun, K. Yao, Q. Cai, J. Shen, Y. Tian, and X Zhou 2022, Developing deep learning based regression approaches for prediction of firmness and pH in Kyoho grape using Vis/NIR hyperspectral imaging. Infrared Phys Technol 120:104003. doi:10.1016/j.infrared.2021.104003 10.1016/j.infrared.2021.104003
24
Yu X., H. Lu, and D. Wu 2018, Development of deep learning method for predicting firmness and soluble solid content of postharvest Korla fragrant pear using Vis/NIR hyperspectral reflectance imaging. Postharvest Biol Technol 141:39-49. doi:10.1016/j.postharvbio.2018.02.013 10.1016/j.postharvbio.2018.02.013
Information
  • Publisher :The Korean Society for Bio-Environment Control
  • Publisher(Ko) :(사)한국생물환경조절학회
  • Journal Title :Journal of Bio-Environment Control
  • Journal Title(Ko) :생물환경조절학회지
  • Volume : 31
  • No :4
  • Pages :270-278
  • Received Date : 2022-07-04
  • Revised Date : 2022-08-17
  • Accepted Date : 2022-09-10
  • DOI :https://doi.org/10.12791/KSBEC.2022.31.4.270
Journal Informaiton Journal of Bio-Environment Control Journal of Bio-Environment Control
Online Submission
  • NRF
  • KOFST
  • crossref crossmark
  • crossref cited-by
  • crossref funder-registry
  • crosscheck
  • orcid
  • open access
  • ccl
Journal Informaiton Journal Informaiton - close