How Robust Are Character-Based Word Embeddings in Tagging and MT Against Wrod Scramlbing or Randdm Nouse?
Preprint
Abstract
This paper investigates the robustness of NLP against perturbed word forms. While neural approaches can achieve (almost) human-like accuracy for certain tasks and conditions, they often are sensitive to small changes in the input such as non-canonical input (e.g., typos). Yet both stability and robustness are desired properties in applications involving user-generated content, and the more as humans easily cope with such noisy or adversary conditions. In this paper, we study the impact of noisy input. We consider different noise distributions (one type of noise, combination of noise types) and mismatched noise distributions for training and testing. Moreover, we empirically evaluate the robustness of different models (convolutional neural networks, recurrent neural networks, non-neural models), different basic units (characters, byte pair encoding units), and different NLP tasks (morphological tagging, machine translation).Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- An Extensive Empirical Evaluation of Character-Based Morphological Tagging for 14 Languages2017
- How Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairs2017
- Learning Robust Representations of Text2016
- A Character-level Decoder without Explicit Segmentation for Neural Machine Translation2016
- Multilingual Language Processing From Bytes2016
- Neural versus Phrase-Based Machine Translation Quality: a Case Study2016
- Data Augmentation for Deep Neural Network Acoustic ModelingIEEE/ACM Transactions on Audio, Speech, and Language Processing, 2015
- The Significance of Letter Position in Word RecognitionIEEE Aerospace and Electronic Systems Magazine, 2007
- Multi-style training for robust isolated-word speech recognition2005
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998