Deep Learning Models for Document Classification

Abstract

Document classification is an essential process of natural language processing (NLP) which presupposes assigning textual documents to predefined categories depending on their contents. As the amount of digital text that needs to be classified has grown exponentially through the sources of social media, scholarly repositories, law archives, news portals and enterprise document management systems, effective and correct document classification has become more important. Most of the common machine learning models such as Naïve Bayes, Support Vector Machines, and k-Nearest Neighbors have proven to be of acceptable performance but they heavily depend on manually crafted features and are not as accurate in detecting semantic and contextual information in text. The latest technology in deep learning greatly altered the methods of document classification as it allowed extracting features and learning representations based on the context. Convolutional neural networks (CNNs) models, recurrent neural networks (RNNs), Long Short Memory networks (LSTMs), Gated Recurrent Units (GRUs) and transformer-based have been used to set the state of the art on benchmark datasets. Such models employ dense word encodings, attention, and hierarchical models to represent document-level semantic structures on both local and global levels. In the present paper, the systematic investigation of the document classification frameworks using deep learning models is offered. It analyzes background information, architectural design, learning process and optimization plans. Moreover, it evaluates the advantages and weaknesses of the different deep learning methods in processing long texts, multi-label classification, domain adaptation, and scalability. The socio-economic metrics are also standard performance metrics, and a single approach to the methodology is suggested that incorporates preprocessing, embedding learning, model training, and evaluation using these metrics. The results of the experiment when exploring representative datasets are addressed to emphasize the trends of the comparative performance. The paper will end by presenting some of the current challenges and the direction of future research and stressing the aspects of explainability, efficiency, and domain robustness.