Multimodal Music Mood Classification Framework for Kokborok Music

Abstract

This article describes one of the applications of Music information retrieval (MIR) integrated with natural language processing. The proposed work represents one of the applications of MIR that is music mood classification of one of the North-eastern regional language, which is Kokborok. It is widely spoken in the states of North East (NE) India and many other countries like Nepal, Bhutan, Myanmar and Bangladesh. The selection of the song is particular to Kokborok songs collected from the Bible, which has written in the recognized Romanized language which is accepted worldwide. We develop the multimodal corpus for audio and lyrics for Kokborok song and performed coarse-grained annotation to create mood annotated dataset and then perform classification task on both audio and lyrics separately. We projected mood taxonomy for Kokborok songs and set a mood annotated corpus with the corresponding taxonomy. Initially, we used 48 parameters for audio classification and six Text stylistic feature for lyrics based classification. The SVM classifier is used with linear kernel function for classification.  Finally, Mood classification system was developed for Kokborok song consist of three different systems based on audio, lyrics and multimodal (audio and lyrics together). We also compared different classifier used to get the system performance for the above three systems. We achieved 95% accuracy for audio, 97% for lyrics and multimodal system, and the accuracy rate is about 96%.