PENERAPAN CONVOLUTIONAL NEURAL NETWORK DALAM ANALISIS EMOSI

Zelvi Gustiana; Welnof Satria

doi:10.46576/djtechno.v5i3.5693

PENERAPAN CONVOLUTIONAL NEURAL NETWORK DALAM ANALISIS EMOSI

Zelvi Gustiana, Welnof Satria

Abstract

Pengenalan emosi memainkan peran penting dalam berbagai aplikasi, seperti interaksi manusia-mesin, layanan kesehatan mental, dan sistem pengajaran adaptif. Convolutional Neural Network (CNN) telah menjadi metode yang andal untuk pengenalan emosi karena kemampuannya dalam mengenali pola kompleks dari data visual, audio, dan multimodal. Penelitian ini bertujuan untuk mengkaji penerapan CNN dalam pengenalan emosi, termasuk jenis dataset yang digunakan, arsitektur model, metode augmentasi data, dan tantangan implementasi.

Hasil kajian menunjukkan bahwa CNN mampu mencapai akurasi tinggi pada dataset seperti FER2013, CK+, dan RAVDESS, dengan rata-rata akurasi di atas 80%. Metode augmentasi data, seperti rotasi, flipping, dan penyesuaian pencahayaan, membantu meningkatkan generalisasi model. Namun, penelitian ini masih menghadapi tantangan, termasuk keterbatasan dataset yang kurang representatif, kebutuhan komputasi yang tinggi, dan kurangnya interpretabilitas model. Pendekatan multimodal yang menggabungkan data citra, suara, dan teks juga menunjukkan hasil yang menjanjikan dengan akurasi hingga 92%.

Penelitian ini menyimpulkan bahwa CNN memiliki potensi besar untuk mendukung pengembangan sistem pengenalan emosi yang lebih akurat dan andal. Pengembangan dataset yang lebih inklusif, integrasi metode mutakhir, dan penerapan teknik explainable AI direkomendasikan untuk penelitian di masa depan.

Keywords

Pengenalan Emosi, Convolutional Neural Network, Augmentasi Data, Multimodal, Interpretabilitas.

Full Text:

PDF

References

A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," Communications of the ACM, vol. 60, no. 6, pp. 84–90, June 2017, doi: 10.1145/3065386.

B. Zhang, W. Li, and P. Shen, "Facial expression recognition using sparse representation and local fisher discriminant analysis," Visual Computer, vol. 29, no. 9, pp. 879–888, Sep. 2013, doi: 10.1007/s00371-012-0752-0.

D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by back-propagating errors," Nature, vol. 323, pp. 533–536, 1986, doi: 10.1038/323533a0.

E. Parry, E. Hudson, and M. Tang, "Analysis of speech emotion recognition using deep learning techniques," Journal of Artificial Intelligence and Soft Computing Research, vol. 10, no. 4, pp. 273–283, Oct. 2020, doi: 10.2478/jaiscr-2020-0019.

F. Chollet, "Xception: Deep learning with depthwise separable convolutions," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 1251–1258, doi: 10.1109/CVPR.2017.195.

G. Levi and T. Hassner, "Emotion recognition in the wild via convolutional neural networks and mapped binary patterns," Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (ICMI), 2015, pp. 503–510, doi: 10.1145/2818346.2830595.

G. Zhao and M. Pietikäinen, "Dynamic texture recognition using local binary patterns with an application to facial expressions," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 6, pp. 915–928, June 2007, doi: 10.1109/TPAMI.2007.1110.

H. Kaya, F. Gürpinar, and A. A. Salah, "Video-based emotion recognition in the wild using deep transfer learning and score fusion," Image and Vision Computing, vol. 65, pp. 66–75, Aug. 2017, doi: 10.1016/j.imavis.2017.01.012.

K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778, doi: 10.1109/CVPR.2016.90.

K. Zhang, Z. Huang, D. Zhang, and H. Yu, "Attention-based convolutional neural networks for speech emotion recognition," IEEE Access, vol. 7, pp. 71258–71269, 2019, doi: 10.1109/ACCESS.2019.2919065.

M. E. Ayadi, M. S. Kamel, and F. Karray, "Survey on speech emotion recognition: Features, classification schemes, and databases," Pattern Recognition, vol. 44, no. 3, pp. 572–587, Mar. 2011, doi: 10.1016/j.patcog.2010.09.020.

M. Pantic and L. Rothkrantz, "Automatic analysis of facial expressions: The state of the art," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1424–1445, Dec. 2000, doi: 10.1109/34.895976.

M. Z. Uddin, J. D. Lee, and T. Kim, "Speech emotion recognition using deep dense and bidirectional LSTM networks," Sensors, vol. 20, no. 9, p. 2776, Apr. 2020, doi: 10.3390/s20092776.

Mollahosseini, D. Chan, and M. H. Mahoor, "Going deeper in facial expression recognition using deep neural networks," 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), 2016, pp. 1–10, doi: 10.1109/WACV.2016.7477450.

P. Tzirakis, J. Zhang, and B. W. Schuller, "End-to-end speech emotion recognition using deep neural networks," Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, pp. 5084–5088, doi: 10.1109/ICASSP.2018.8461329.

R. Wang, J. Tao, and Y. Tian, "Multimodal speech emotion recognition using deep convolutional neural networks," IEEE Transactions on Affective Computing, vol. 12, no. 1, pp. 236–246, Jan.–Mar. 2021, doi: 10.1109/TAFFC.2019.2928365.

S. Li and W. Deng, "Deep facial expression recognition: A survey," IEEE Transactions on Affective Computing, vol. 13, no. 1, pp. 119–135, Jan. 2022, doi: 10.1109/TAFFC.2020.2981446.

S. Minaee, A. Abtahi, and Y. Wang, "Facial expression recognition using feature fusion of 2D and 3D data," IEEE Signal Processing Letters, vol. 23, no. 5, pp. 610–614, May 2016, doi: 10.1109/LSP.2016.2537278.

T. Baltrusaitis, C. Ahuja, and L. P. Morency, "Multimodal machine learning: A survey and taxonomy," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 2, pp. 423–443, Feb. 2019, doi: 10.1109/TPAMI.2018.2798607.

Z. Zhou, G. Zhao, and X. Ding, "Multimodal emotion recognition based on transfer learning using deep convolutional neural networks," Information Fusion, vol. 76, pp. 79–92, Nov. 2021, doi: 10.1016/j.inffus.2021.05.004.

DOI: https://doi.org/10.46576/djtechno.v5i3.5693