Sentiment analysis using attention-based convolution autoencoder (SAABCA)

Neha Sharma; S Veenadhari

Authors

Neha Sharma Department of CSE, Rabindranath Tagore University (RNTU), Bhopal (M.P), India
S Veenadhari Department of CSE, Rabindranath Tagore University (RNTU), Bhopal (M.P), India

Keywords:

Sentiment Analysis, Deep Learning, Autoencoder, Attention based Convolution Autoencoder.

Abstract

Sentiment analysis (SA) has been a commonly studied issue in the disciplines of NLP, data mining and data analysis. Deep neural network (DNN) algorithms have lately been used to do SA with significant improvements. However, these algorithms can handle sequencing of any lengths, employing it in the extracting features of a Deep Neural Network increases the dimensionality of the feature space. In this paper, a sentimental analysis using attention-based convolution autoencoder (SAABCA) model is proposed to tackle such challenges. Moreover, the Attention mechanism (AM) is used on the outcomes of layers. SAABCA employs convolutional and pooling methods to decrease feature dimensions and recover position-invariant feature points. The efficacy of model is measured by its ability to identify sentiment orientation, that is the most popular and essential job in SA. As contrasted to seven previously recommended DNNs for sentiment analysis, additional state-of-the-art effectiveness is shown on several review classifications and tweet polarity classifications. The findings indicate that the suggested technique accuracy is 94% for Kindle Dataset, 97% for Movie Dataset and 98% for Airline Twitter Dataset.

References

Balas, V. E., Roy, S. S., Sharma, D., & Samui, P. (Eds.). (2019). Handbook of deep learning applications (Vol. 136). New York: Springer. DOI: https://doi.org/10.1007/978-3-030-11479-4

Basiri, M. E., Nemati, S., Abdar, M., Cambria, E., & Acharya, U. R. (2021). ABCDM: An attention-based bidirectional CNN-RNN deep model for sentiment analysis. Future Generation Computer Systems, 115, 279-294. DOI: https://doi.org/10.1016/J.FUTURE.2020.08.005

Chatterjee, A., Gupta, U., Chinnakotla, M. K., Srikanth, R., Galley, M., & Agrawal, P. (2019). Understanding emotions in text using deep learning and big data. Computers in Human Behavior, 93, 309-317. DOI: https://doi.org/10.1016/J.CHB.2018.12.029

Chaturvedi, I., Ragusa, E., Gastaldo, P., Zunino, R., & Cambria, E. (2018). Bayesian network based extreme learning machine for subjectivity detection. Journal of the Franklin Institute, 355(4), 1780-1797. DOI: https://doi.org/10.1016/J.JFRANKLIN.2017.06.007

Data.word. (2019). Airline twitter sentiment dataset. Retrieve form https://data.world/crowdflower/airline-twitter-sentiment.

Dey, S., Wasif, S., Tonmoy, D. S., Sultana, S., Sarkar, J., & Dey, M. (2020, February). A comparative study of support vector machine and Naive Bayes classifier for sentiment analysis on Amazon product reviews. In 2020 International Conference on Contemporary Computing and Applications (IC3A) (pp. 217-220). IEEE. DOI: 10.1109/IC3A48958.2020.233300

El-Affendi, M. A., Alrajhi, K., & Hussain, A. (2021). A novel deep learning-based multilevel parallel attention neural (MPAN) model for multidomain arabic sentiment analysis. IEEE Access, 9, 7508-7518. DOI: 10.1109/ACCESS.2021.3049626.

Fu, X., Liu, W., Xu, Y., & Cui, L. (2017). Combine HowNet lexicon to train phrase recursive autoencoder for sentence-level sentiment analysis. Neurocomputing, 241, 18-27. https://doi.org/10.1016/J.NEUCOM.2017.01.079

Gan, C., Wang, L., & Zhang, Z. (2020). Multi-entity sentiment analysis using self-attention based hierarchical dilated convolutional neural network. Undefined, 112, 116–125. DOI: https://doi.org/ Gan 10.1016/J.FUTURE.2020.05.022

Gautam, J., Atrey, M., Malsa, N., Balyan, A., Shaw, R. N., & Ghosh, A. (2021). Twitter data sentiment analysis using naive bayes classifier and generation of heat map for analyzing intensity geographically. In Advances in Applications of Data-Driven Computing (pp. 129-139). Springer, Singapore. DOI: https://doi.org/10.1007/978-981-33-6919-1_10

Guellil, I., Adeel, A., Azouaou, F., Benali, F., Hachani, A. E., Dashtipour, K., ... & Hussain, A. (2021). A semi-supervised approach for sentiment analysis of arab (ic+ izi) messages: Application to the algerian dialect. SN Computer Science, 2(2), 1-18. DOI: https://doi.org/10.1007/s42979-021-00510-1

He, R., & McAuley, J. (2016, April). Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In proceedings of the 25th international conference on world wide web (pp. 507-517).

Hossen, M., & Dev, N. R. (2021). An improved lexicon based model for efficient sentiment analysis on movie review data. Wireless Personal Communications, 120(1), 535-544. DOI: https://doi.org/10.1007/s11277-021-08474-4

Jigneshkumar Patel, H., Prakash Verma, J., & Patel, A. (2021). Unsupervised Learning-Based Sentiment Analysis with Reviewer’s Emotion. In Evolving Technologies for Computing, Communication and Smart World (pp. 69-81). Springer, Singapore. DOI: https://doi.org/10.1007/978-981-15-7804-5_6

Kaul, C., Manandhar, S., & Pears, N. (2019, April). Focusnet: An attention-based fully convolutional network for medical image segmentation. In 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019) (pp. 455-458). IEEE. DOI: https://doi.org/10.1109/ISBI.2019.8759477

Kumar, V., & Subba, B. (2020, February). A TfidfVectorizer and SVM based sentiment analysis framework for text data corpus. In 2020 National Conference on Communications (NCC) (pp. 1-6). IEEE. DOI: 10.1109/NCC48643.2020.9056085.

Li, Z., Li, R., & Jin, G. (2020). Sentiment analysis of danmaku videos based on naïve bayes and sentiment dictionary. Ieee Access, 8, 75073-75084. DOI: 10.1109/ACCESS.2020.2986582

Liu, G., & Guo, J. (2019). Bidirectional LSTM with attention mechanism and convolutional layer for text classification. Neurocomputing, 337, 325-338. DOI: https://doi.org/10.1016/J.NEUCOM.2019.01.078

Luc Phan, L., Huynh Pham, P., Thi-Thanh Nguyen, K., Khai Huynh, S., Thi Nguyen, T., Thanh Nguyen, L., ... & Van Nguyen, K. (2021, August). SA2SL: From Aspect-Based Sentiment Analysis to Social Listening System for Business Intelligence. In International Conference on Knowledge Science, Engineering and Management (pp. 647-658). Springer, Cham. DOI: https://doi.org/10.1007/978-3-030-82147-0_53

Luo, J., Huang, S., & Wang, R. (2021). A fine-grained sentiment analysis of online guest reviews of economy hotels in China. Journal of Hospitality Marketing & Management, 30(1), 71-95.

Mack, J., Arcucci, R., Molina-Solana, M., & Guo, Y. K. (2020). Attention-based Convolutional Autoencoders for 3D-Variational Data Assimilation. Computer Methods in Applied Mechanics and Engineering, 372, 113291. DOI: https://doi.org/10.1016/J.CMA.2020.113291

McAuley, J., Targett, C., Shi, Q., & Van Den Hengel, A. (2015, August). Image-based recommendations on styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval (pp. 43-52).

Onan, A. (2015). A fuzzy-rough nearest neighbor classifier combined with consistency-based subset evaluation and instance selection for automated diagnosis of breast cancer. Expert Systems with Applications, 42(20), 6844–6852. DOI: https://doi.org/10.1016/J.ESWA.2015.05.006

Onan, A. (2016). Classifier and feature set ensembles for web page classification. Journal of Information Science, 42(2), 150-165. DOI: https://doi.org/10.1177/0165551515591724

Onan, A. (2017). Hybrid supervised clustering based ensemble scheme for text classification. Kybernetes, 46(2), 330-348. DOI: https://doi.org/10.1108/K-10-2016-0300/FULL/XML

Onan, A. (2018). An ensemble scheme based on language function analysis and feature engineering for text genre classification. Journal of Information Science, 44(1), 28-47. DOI: https://doi.org/10.1177/0165551516677911

Onan, A. (2019). Two-stage topic extraction model for bibliometric data analysis based on word embeddings and clustering. IEEE Access, 7, 145614-145633. DOI: https://doi.org/10.1109/ACCESS.2019.2945911

Onan, A. (2019a). Consensus Clustering-Based Undersampling Approach to Imbalanced Learning. Scientific Programming, 2019. DOI: https://doi.org/10.1155/2019/5901087

Onan, A. (2019b). Topic-enriched word embeddings for sarcasm identification. In Computer science on-line conference (pp. 293-304). Springer, Cham. DOI: https://doi.org/10.1007/978-3-030-19807-7_29

Onan, A. (2020). Mining opinions from instructor evaluation reviews: A deep learning approach. Computer Applications in Engineering Education, 28(1), 117-138. DOI: https://doi.org/10.1002/CAE.22179

Onan, A. (2021). Sentiment analysis on product reviews based on weighted word embeddings and deep neural networks. Concurrency and Computation: Practice and Experience, 33(23), e5909. DOI: https://doi.org/10.1002/CPE.5909

Onan, A., & KorukoGlu, S. (2017). A feature selection model based on genetic rank aggregation for text sentiment classification. Journal of Information Science, 43(1), 25-38. DOI: https://doi.org/10.1177/0165551515613226

Onan, A., & Toçoğlu, M. A. (2021). A term weighted neural language model and stacked bidirectional LSTM based framework for sarcasm identification. IEEE Access, 9, 7701-7722. DOI: https://doi.org/10.1109/ACCESS.2021.3049734

Onan, A., Bal, V., & Yanar Bayam, B. (2016). The use of data mining for strategic management: a case study on mining association rules in student information system. Croatian Journal of Education: Hrvatski časopis za odgoj i obrazovanje, 18(1), 41-70. DOI: https://doi.org/10.15516/CJE.V18I1.1471

Onan, A., Korukoğlu, S., & Bulut, H. (2016). Ensemble of keyword extraction methods and classifiers in text classification. Expert Systems with Applications, 57, 232-247. DOI: https://doi.org/10.1016/J.ESWA.2016.03.045

Poornima, A., & Priya, K. S. (2020, March). A comparative sentiment analysis of sentence embedding using machine learning techniques. In 2020 6th international conference on advanced computing and communication systems (ICACCS) (pp. 493-496). IEEE. DOI: 10.1109/ICACCS48705.2020.9074312

Rahman, M., & Islam, M. N. (2022). Exploring the performance of ensemble machine learning classifiers for sentiment analysis of covid-19 tweets. In Sentimental Analysis and Deep Learning (pp. 383-396). Springer, Singapore. DOI: https://doi.org/10.1007/978-981-16-5157-1_30

Rezaeinia, S. M., Rahmani, R., Ghodsi, A., & Veisi, H. (2019). Sentiment analysis based on improved pre-trained word embeddings. Expert Systems with Applications, 117, 139–147. DOI: https://doi.org/10.1016/J.ESWA.2018.08.044

Rice, D., & Zorn, C. (2021). Corpus-based dictionaries for sentiment analysis of specialized vocabularies. Political Science Research and Methods, 9(1), 20-35. DOI: 10.1017/psrm.2019.10

Roy, S. S., Biba, M., Kumar, R., Kumar, R., & Samui, P. (2017). A new SVM method for recognizing polarity of sentiments in twitter. In Handbook of Research on Soft Computing and Nature-Inspired Algorithms (pp. 281-291). IGI Global.

Ruz, G. A., Henríquez, P. A., & Mascareño, A. (2020). Sentiment analysis of Twitter data during critical events through Bayesian networks classifiers. Future Generation Computer Systems, 106, 92-104. DOI: https://doi.org/10.1016/j.future.2020.01.005

Seçkin, T., & Kilimci, Z. H. (2020, October). The evaluation of 5G technology from sentiment analysis perspective in Twitter. In 2020 Innovations in Intelligent Systems and Applications Conference (ASYU) (pp. 1-6). IEEE. DOI: 10.1109/ASYU50717.2020.9259900

Sidhu, S., & Khurana, S. S. (2022). Method to Rank Academic Institutes by the Sentiment Analysis of Their Online Reviews. In Research Anthology on Implementing Sentiment Analysis Across Multiple Disciplines (pp. 555-580). IGI Global.

Softic, S., & Lüftenegger, E. (2022). Towards Empowering Business Process Redesign with Sentiment Analysis. In Proceedings of Sixth International Congress on Information and Communication Technology (pp. 119-126). Springer, Singapore. DOI: https://doi.org/10.1007/978-981-16-2380-6_10

Toçoğlu, M. A., & Onan, A. (2020, July). Sentiment analysis on students’ evaluation of higher educational institutions. In International Conference on Intelligent and Fuzzy Systems (pp. 1693-1700). Springer, Cham. DOI: https://doi.org/10.1007/978-3-030-51156-2_197

Wang, J., Yu, L. C., Lai, K. R., & Zhang, X. (2016, August). Dimensional sentiment analysis using a regional CNN-LSTM model. In Proceedings of the 54th annual meeting of the association for computational linguistics (volume 2: Short papers) (pp. 225-230). DOI: https://doi.org/10.18653/V1/P16-2037

Wen, S., & Li, J. (2018). Recurrent convolutional neural network with attention for twitter and yelp sentiment classification arc model for sentiment classification. ACM International Conference Proceeding Series. DOI: https://doi.org/10.1145/3302425.3302468

Wu, C., Wu, F., Wu, S., Yuan, Z., Liu, J., & Huang, Y. (2019). Semi-supervised dimensional sentiment analysis with variational autoencoder. Knowledge-Based Systems, 165, 30-39. DOI: https://doi.org/10.1016/J.KNOSYS.2018.11.018.

Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016, June). Hierarchical attention networks for document classification. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies (pp. 1480-1489).