Intelligent Document Classification Using Neural Networks

  • Authors

    • Dr. Kwame Nkosi Department of Economics, University of Ghana, Accra, Ghana. Author

    DOI:

    https://doi.org/10.67228/30713498/IJADSMC-2019PI9L7W

    Published 07-02-2019

  • Document Classification, Neural Networks, Deep Learning, Natural Language Processing, Text Mining

    Issue

    Vol. 2 No. 2 (2019)

    Section

    Articles

    How to Cite

    [1]
    K. Nkosi, “Intelligent Document Classification Using Neural Networks”, IJADSMC, vol. 2, no. 2, pp. 01–12, Jul. 2019, doi: 10.67228/30713498/IJADSMC-2019PI9L7W.

  • Abstract

    Subsequent to smart document categorization has become a core operation in contemporary information discovery, online libraries, business information management and big-data analytics. As the amount of unstructured textual information in terms of academic repositories, social media sites, corporate archives and government databases grows exponentially, automated and intelligent classification methods are critical in terms of efficient data organization and knowledge discovery. Although effective in limited situations, traditional rule-based and statistical machine learning methods fail to scale and generalize when faced with semantic ambiguity, contextual differences and high-dimensional feature spaces. Neural networks and especially deep learning models have proven themselves able to extract semantic representations and contextual relationships in text data remarkably. In this paper, the intelligent document classification using neural networks will be studied in a very comprehensive manner in terms of the architecture, features representation, training techniques, and evaluation procedures. The framework proposed combines text preprocessing, embedding, and neural classification models to obtain robust and scalable document classification. An overall experimental study is performed based on benchmark datasets in an attempt to determine the accuracy of classification, the precision, recall, and the computational efficiency. The findings show that neural network methods are very effective compared to traditional methods particularly when dealing with large and complicated document collections. The paper concludes by mentioning practical implications, challenges, and future research directions in the area of neural document classification.

  • References

    [1] Joachims, T. (1998). Text categorization with Support Vector Machines: Learning with many relevant features. Proceedings of ECML, 137–142.

    [2] McCallum, A., & Nigam, K. (1998). A comparison of event models for Naïve Bayes text classification. AAAI Workshop on Learning for Text Categorization, 41–48.

    [3] Salton, G., & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Information Processing & Management, 24(5), 513–523.

    [4] Sebastiani, F. (2002). Machine learning in automated text categorization. ACM Computing Surveys, 34(1), 1–47.

    [5] Vapnik, V. N. (1995). The nature of statistical learning theory. Springer-Verlag.

    [6] Bishop, C. M. (2006). Pattern recognition and machine learning. Springer.

    [7] Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. (2011). Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12, 2493–2537.

    [8] Kim, Y. (2014). Convolutional neural networks for sentence classification. Proceedings of EMNLP, 1746–1751.

    [9] Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.

    [10] Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.

    [11] Graves, A. (2012). Supervised sequence labelling with recurrent neural networks. Springer.

    [12] Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention is all you need. Advances in Neural Information Processing Systems (NeurIPS), 5998–6008.

    [13] Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACL-HLT, 4171–4186.

    [14] Liu, Y., Ott, M., Goyal, N., et al. (2019). RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692.

    [15] Jain, S., & Wallace, B. C. (2019). Attention is not explanation. Proceedings of NAACL-HLT, 3543–3556.

  • Downloads