At my work, I’ve been actively working on text classification problems. I began with simple Random Forest based model and now switched to using a hierarchical deep neural network for a domain specific problem. Meanwhile, I’ve been investigating a number of approaches which I’ve tested empirically and seem to work at large scale. Here are a few papers worth mentioning.
- Yoon Kim, Convolutional Neural Networks for sentence classification [arXiv:1408.5882]
- Yang et. al., Hierarchical Attention Networks for Document Classification [CMU link]
- Liu et. al, Deep Learning for Extreme Multi-label Text Classification [ACM]
- Johnson et. al., Deep Pyramid Convolutional Neural Networks for Text Categorization [Tencent AI]
Until now, for my case, Kim’s CNN approach has been the fastest approach in terms of training for me. This gives approximately the same accuracy as a hierarchical deep neural network that I trained earlier. Probably, the next thing I’m going to try is to add attention mechanism to improve the accuracy of the overall model.