Portal de Periódicos da CAPES

Sentiment analysis and spam filtering using the YAC2 clustering algorithm with transferability

2022; Elsevier BV; Volume: 165; Linguagem: Inglês

10.1016/j.cie.2022.107959

1879-0550

M. Ghiassi, Sean Lee, Swati Ramesh Gaikwad,

Text and Document Classification Technologies

Two notable applications of text classification are sentiment analysis and spam filtering. Traditional machine learning approaches to text classification are often complex, non-transferrable, and require supervision. This paper introduces an unsupervised approach to text classification which is relatively simple and transfers between problem domains, while providing accuracy comparable or better than established alternatives. We present an integrated solution which combines a new clustering algorithm, Yet Another Clustering Algorithm (YAC2), with a domain transferrable feature engineering approach for Twitter sentiment analysis and spam filtering of YouTube comments. We evaluate the effectiveness of this integrated solution for Twitter sentiment analysis using three datasets: Starbucks, Verizon, and Southwest Airlines. YouTube spam filtering is evaluated using four datasets: Psy, LMFAO, Shakira, and Katy Perry. We compare the results with established clustering solutions: KNN, Spectral, and DBSCAN. Our integrated solution performs better than all the alternatives for sentiment analysis. For spam filtering, YAC2 and KNN perform within 1% of each other and far better than Spectral and DBSCAN for all datasets. Additionally, our feature engineering approach improves accuracy compared to using a traditional method, while significantly reducing model dimensionality, matrix sparsity and providing transferability across the datasets tested.