Masahiro Kaneko
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Masahiro Kaneko has a total of 13 co-authors. Here are the top co-authors by number of shared publications:
- Kentaro Inui
Tohoku University
2 shared publications - Mamoru Komachi
Hitotsubashi University, 2-1 Naka, Kunitachi, Tokyo 186-8601, Japan
2 shared publications Masato Mita
RIKEN Center for Advanced Intelligence Project
2 shared publications- Aizhan Imankulova
Tokyo Metropolitan University
1 shared publication - Danushka Bollegala
Department of Computer Science, University of Liverpool, Liverpool, United Kingdom
1 shared publication Jun Suzuki
RIKEN Center for Advanced Intelligence Project
1 shared publication
Latest Publications
- Open Access
Autoencoding Improves Pre-trained Word Embeddings
Prior work investigating the geometry of pre-trained word embeddings have shown that word embeddings to be distributed in a narrow cone and by centering and projecting using principal component vectors one can increase the accuracy of a given set of pre-trained word embeddings. However, theoretically, this post-processing step is equivalent to applying a linear autoencoder to minimise the squared l2 reconstruction error. This result contradicts prior work (Mu and Viswanath, 2018) that proposed to remove the top principal components from pre-trained embeddings. We experimentally verify our theoretical claims and show that retaining the top principal components is indeed useful for improving pre-trained word embeddings, without requiring access to additional linguistic resources or labelled data.
Open AccessAutoencoding Improves Pre-trained Word Embeddings
Prior work investigating the geometry of pre-trained word embeddings have shown that word embeddings to be distributed in a narrow cone and by centering and projecting using principal component vectors one can increase the accuracy of a given set of pre-trained word embeddings. However, theoretically, this post-processing step is equivalent to applying a linear autoencoder to minimise the squared l2 reconstruction error. This result contradicts prior work (Mu and Viswanath, 2018) that proposed to remove the top principal components from pre-trained embeddings. We experimentally verify our theoretical claims and show that retaining the top principal components is indeed useful for improving pre-trained word embeddings, without requiring access to additional linguistic resources or labelled data.
- Open Access
A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction
Existing approaches for grammatical error correction (GEC) largely rely on supervised learning with manually created GEC datasets. However, there has been little focus on verifying and ensuring the quality of the datasets, and on how lower-quality data might affect GEC performance. We indeed found that there is a non-negligible amount of "noise" where errors were inappropriately edited or left uncorrected. To address this, we designed a self-refinement method where the key idea is to denoise these datasets by leveraging the prediction consistency of existing models, and outperformed strong denoising baseline methods. We further applied task-specific techniques and achieved state-of-the-art performance on the CoNLL-2014, JFLEG, and BEA-2019 benchmarks. We then analyzed the effect of the proposed denoising method, and found that our approach leads to improved coverage of corrections and facilitated fluency edits which are reflected in higher recall and overall performance.
Open AccessA Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction
Existing approaches for grammatical error correction (GEC) largely rely on supervised learning with manually created GEC datasets. However, there has been little focus on verifying and ensuring the quality of the datasets, and on how lower-quality data might affect GEC performance. We indeed found that there is a non-negligible amount of "noise" where errors were inappropriately edited or left uncorrected. To address this, we designed a self-refinement method where the key idea is to denoise these datasets by leveraging the prediction consistency of existing models, and outperformed strong denoising baseline methods. We further applied task-specific techniques and achieved state-of-the-art performance on the CoNLL-2014, JFLEG, and BEA-2019 benchmarks. We then analyzed the effect of the proposed denoising method, and found that our approach leads to improved coverage of corrections and facilitated fluency edits which are reflected in higher recall and overall performance.
- Open Access
Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction
This paper investigates how to effectively incorporate a pre-trained masked language model (MLM), such as BERT, into an encoder-decoder (EncDec) model for grammatical error correction (GEC). The answer to this question is not as straightforward as one might expect because the previous common methods for incorporating a MLM into an EncDec model have potential drawbacks when applied to GEC. For example, the distribution of the inputs to a GEC model can be considerably different (erroneous, clumsy, etc.) from that of the corpora used for pre-training MLMs; however, this issue is not addressed in the previous methods. Our experiments show that our proposed method, where we first fine-tune a MLM with a given GEC corpus and then use the output of the fine-tuned MLM as additional features in the GEC model, maximizes the benefit of the MLM. The best-performing model achieves state-of-the-art performances on the BEA-2019 and CoNLL-2014 benchmarks. Our code is publicly available at: https://github.com/kanekomasahiro/bert-gec.
Open AccessEncoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correction
This paper investigates how to effectively incorporate a pre-trained masked language model (MLM), such as BERT, into an encoder-decoder (EncDec) model for grammatical error correction (GEC). The answer to this question is not as straightforward as one might expect because the previous common methods for incorporating a MLM into an EncDec model have potential drawbacks when applied to GEC. For example, the distribution of the inputs to a GEC model can be considerably different (erroneous, clumsy, etc.) from that of the corpora used for pre-training MLMs; however, this issue is not addressed in the previous methods. Our experiments show that our proposed method, where we first fine-tune a MLM with a given GEC corpus and then use the output of the fine-tuned MLM as additional features in the GEC model, maximizes the benefit of the MLM. The best-performing model achieves state-of-the-art performances on the BEA-2019 and CoNLL-2014 benchmarks. Our code is publicly available at: https://github.com/kanekomasahiro/bert-gec.
- Open Access
- 1 January 2020
- Published by Association for Computational Linguistics (ACL)
- p. 594-603
Towards Multimodal Simultaneous Neural Machine Translation
Aizhan Imankulova, Masahiro Kaneko, Tosho Hirasawa, Mamoru Komachi. Proceedings of the Fifth Conference on Machine Translation. 2020.
- 1 January 2020
- Published by Association for Computational Linguistics (ACL)
- p. 594-603
Open AccessTowards Multimodal Simultaneous Neural Machine Translation
Aizhan Imankulova, Masahiro Kaneko, Tosho Hirasawa, Mamoru Komachi. Proceedings of the Fifth Conference on Machine Translation. 2020.
- Open Access
- 1 January 2020
- Published by Association for Computational Linguistics (ACL)
- p. 58-63
Translation of New Named Entities from English to Chinese
Zizheng Zhang, Tosho Hirasawa, Wei Houjing, Masahiro Kaneko, Mamoru Komachi. Proceedings of the 7th Workshop on Asian Translation. 2020.
- 1 January 2020
- Published by Association for Computational Linguistics (ACL)
- p. 58-63
Open AccessTranslation of New Named Entities from English to Chinese
Zizheng Zhang, Tosho Hirasawa, Wei Houjing, Masahiro Kaneko, Mamoru Komachi. Proceedings of the 7th Workshop on Asian Translation. 2020.