Mark Johnson
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Mark Johnson has a total of 216 co-authors. Here are the top co-authors by number of shared publications:
- Dat Quoc Nguyen
VinAI
12 shared publications Eugene Charniak
11 shared publications- Mark Dras
Macquarie University, Australia
7 shared publications Stephen Gould
6 shared publications- Lan Du
Monash University
5 shared publications Mark Steedman
5 shared publications
Latest Publications
- Open Access
Smoothing Entailment Graphs with Language Models
The diversity and Zipfian frequency distribution of natural language predicates in corpora leads to sparsity in Entailment Graphs (EGs) built by Open Relation Extraction (ORE). EGs are computationally efficient and explainable models of natural language inference, but as symbolic models, they fail if a novel premise or hypothesis vertex is missing at test-time. We present theory and methodology for overcoming such sparsity in symbolic models. First, we introduce a theory of optimal smoothing of EGs by constructing transitive chains. We then demonstrate an efficient, open-domain, and unsupervised smoothing method using an off-the-shelf Language Model to find approximations of missing premise predicates. This improves recall by 25.1 and 16.3 percentage points on two difficult directional entailment datasets, while raising average precision and maintaining model explainability. Further, in a QA task we show that EG smoothing is most useful for answering questions with lesser supporting text, where missing premise predicates are more costly. Finally, controlled experiments with WordNet confirm our theory and show that hypothesis smoothing is difficult, but possible in principle.
Open AccessSmoothing Entailment Graphs with Language Models
The diversity and Zipfian frequency distribution of natural language predicates in corpora leads to sparsity in Entailment Graphs (EGs) built by Open Relation Extraction (ORE). EGs are computationally efficient and explainable models of natural language inference, but as symbolic models, they fail if a novel premise or hypothesis vertex is missing at test-time. We present theory and methodology for overcoming such sparsity in symbolic models. First, we introduce a theory of optimal smoothing of EGs by constructing transitive chains. We then demonstrate an efficient, open-domain, and unsupervised smoothing method using an off-the-shelf Language Model to find approximations of missing premise predicates. This improves recall by 25.1 and 16.3 percentage points on two difficult directional entailment datasets, while raising average precision and maintaining model explainability. Further, in a QA task we show that EG smoothing is most useful for answering questions with lesser supporting text, where missing premise predicates are more costly. Finally, controlled experiments with WordNet confirm our theory and show that hypothesis smoothing is difficult, but possible in principle.
- Open Access
- 1 January 2023
- Published by Association for Computational Linguistics (ACL)
- p. 551-563
Smoothing Entailment Graphs with Language Models
Nick McKenna, Tianyi Li, Mark Johnson, Mark Steedman. Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.
- 1 January 2023
- Published by Association for Computational Linguistics (ACL)
- p. 551-563
Open AccessSmoothing Entailment Graphs with Language Models
Nick McKenna, Tianyi Li, Mark Johnson, Mark Steedman. Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.
- Open Access
- 1 January 2021
- Published by Association for Computational Linguistics (ACL)
- p. 103-113
Mention Flags (MF): Constraining Transformer-based Text Generators
Yufei Wang, Ian Wood, Stephen Wan, Mark Dras, Mark Johnson. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
- 1 January 2021
- Published by Association for Computational Linguistics (ACL)
- p. 103-113
Open AccessMention Flags (MF): Constraining Transformer-based Text Generators
Yufei Wang, Ian Wood, Stephen Wan, Mark Dras, Mark Johnson. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
- Open Access
Neural Constituency Parsing of Speech Transcripts
This paper studies the performance of a neural self-attentive parser on transcribed speech. Speech presents parsing challenges that do not appear in written text, such as the lack of punctuation and the presence of speech disfluencies (including filled pauses, repetitions, corrections, etc.). Disfluencies are especially problematic for conventional syntactic parsers, which typically fail to find any EDITED disfluency nodes at all. This motivated the development of special disfluency detection systems, and special mechanisms added to parsers specifically to handle disfluencies. However, we show here that neural parsers can find EDITED disfluency nodes, and the best neural parsers find them with an accuracy surpassing that of specialized disfluency detection systems, thus making these specialized mechanisms unnecessary. This paper also investigates a modified loss function that puts more weight on EDITED nodes. It also describes tree-transformations that simplify the disfluency detection task by providing alternative encodings of disfluencies and syntactic information.
Open AccessNeural Constituency Parsing of Speech Transcripts
This paper studies the performance of a neural self-attentive parser on transcribed speech. Speech presents parsing challenges that do not appear in written text, such as the lack of punctuation and the presence of speech disfluencies (including filled pauses, repetitions, corrections, etc.). Disfluencies are especially problematic for conventional syntactic parsers, which typically fail to find any EDITED disfluency nodes at all. This motivated the development of special disfluency detection systems, and special mechanisms added to parsers specifically to handle disfluencies. However, we show here that neural parsers can find EDITED disfluency nodes, and the best neural parsers find them with an accuracy surpassing that of specialized disfluency detection systems, thus making these specialized mechanisms unnecessary. This paper also investigates a modified loss function that puts more weight on EDITED nodes. It also describes tree-transformations that simplify the disfluency detection task by providing alternative encodings of disfluencies and syntactic information.
- Open Access
Disfluency Detection using Auto-Correlational Neural Networks
In recent years, the natural language processing community has moved away from task-specific feature engineering, i.e., researchers discovering ad-hoc feature representations for various tasks, in favor of general-purpose methods that learn the input representation by themselves. However, state-of-the-art approaches to disfluency detection in spontaneous speech transcripts currently still depend on an array of hand-crafted features, and other representations derived from the output of pre-existing systems such as language models or dependency parsers. As an alternative, this paper proposes a simple yet effective model for automatic disfluency detection, called an auto-correlational neural network (ACNN). The model uses a convolutional neural network (CNN) and augments it with a new auto-correlation operator at the lowest layer that can capture the kinds of "rough copy" dependencies that are characteristic of repair disfluencies in speech. In experiments, the ACNN model outperforms the baseline CNN on a disfluency detection task with a 5% increase in f-score, which is close to the previous best result on this task.
Open AccessDisfluency Detection using Auto-Correlational Neural Networks
In recent years, the natural language processing community has moved away from task-specific feature engineering, i.e., researchers discovering ad-hoc feature representations for various tasks, in favor of general-purpose methods that learn the input representation by themselves. However, state-of-the-art approaches to disfluency detection in spontaneous speech transcripts currently still depend on an array of hand-crafted features, and other representations derived from the output of pre-existing systems such as language models or dependency parsers. As an alternative, this paper proposes a simple yet effective model for automatic disfluency detection, called an auto-correlational neural network (ACNN). The model uses a convolutional neural network (CNN) and augments it with a new auto-correlation operator at the lowest layer that can capture the kinds of "rough copy" dependencies that are characteristic of repair disfluencies in speech. In experiments, the ACNN model outperforms the baseline CNN on a disfluency detection task with a 5% increase in f-score, which is close to the previous best result on this task.
- 7 February 2020
- Published by SAGE Publications in First Language
- Vol. 40 (5-6) , 573-575
Exemplar-based learning probably requires learning abstractions: A commentary on Ambridge (2020):
Exemplar-based learning requires (i) a segmentation procedure for identifying the units of past experiences that a present experience can be compared to, and (ii) a similarity function for comparing these past experiences to the present experience. This article argues that for a learner to learn a language these two mechanisms will require abstractions such as linguistic features. Moreover, because the segmentation procedure will vary depending on the language, a radical exemplar theory is likely to require linguistic abstractions for learning.
- 7 February 2020
- Published by SAGE Publications in First Language
- Vol. 40 (5-6) , 573-575
Exemplar-based learning probably requires learning abstractions: A commentary on Ambridge (2020):
Exemplar-based learning requires (i) a segmentation procedure for identifying the units of past experiences that a present experience can be compared to, and (ii) a similarity function for comparing these past experiences to the present experience. This article argues that for a learner to learn a language these two mechanisms will require abstractions such as linguistic features. Moreover, because the segmentation procedure will vary depending on the language, a radical exemplar theory is likely to require linguistic abstractions for learning.
- 1 December 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Passive Target Localization by Asynchronous Self-Locating Receivers in Multipath Environments
This paper considers target localization from ambient radio frequency (RF) signals transmitted by the target. We consider a new practical scenario with an array of asynchronous receivers deployed at arbitrary locations, and present an algorithm that jointly locates the target and self-locates the receivers. The approach exploits time-difference-of-arrival (TDoA) between multipath components (multipath TDoA) at each receiver. We derive lower bounds for the localization errors of all the target and receiver locations. The performance is verified numerically and demonstrated experimentally with a hardware implementation that was tested in an anechoic chamber using passive IEEE 802.11ac receivers. We have shown that sub-meter level accuracy can be achieved using 6 static receivers.
- 1 December 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-6
Passive Target Localization by Asynchronous Self-Locating Receivers in Multipath Environments
This paper considers target localization from ambient radio frequency (RF) signals transmitted by the target. We consider a new practical scenario with an array of asynchronous receivers deployed at arbitrary locations, and present an algorithm that jointly locates the target and self-locates the receivers. The approach exploits time-difference-of-arrival (TDoA) between multipath components (multipath TDoA) at each receiver. We derive lower bounds for the localization errors of all the target and receiver locations. The performance is verified numerically and demonstrated experimentally with a hardware implementation that was tested in an anechoic chamber using passive IEEE 802.11ac receivers. We have shown that sub-meter level accuracy can be achieved using 6 static receivers.
- 1 October 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 8947-8956
nocaps: novel object captioning at scale
Image captioning models have achieved impressive results on datasets containing limited visual concepts and large amounts of paired image-caption training data. However, if these models are to ever function in the wild, a much larger variety of visual concepts must be learned, ideally from less supervision. To encourage the development of image captioning models that can learn visual concepts from alternative data sources, such as object detection datasets, we present the first large-scale benchmark for this task. Dubbed ‘nocaps’, for novel object captioning at scale, our benchmark consists of 166,100 human-generated captions describing 15,100 images from the Open Images validation and test sets. The associated training data consists of COCO image-caption pairs, plus Open Images image-level labels and object bounding boxes. Since Open Images contains many more classes than COCO, nearly 400 object classes seen in test images have no or very few associated training captions (hence, nocaps). We extend existing novel object captioning models to establish strong baselines for this benchmark and provide analysis to guide future work.
- 1 October 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 8947-8956
nocaps: novel object captioning at scale
Image captioning models have achieved impressive results on datasets containing limited visual concepts and large amounts of paired image-caption training data. However, if these models are to ever function in the wild, a much larger variety of visual concepts must be learned, ideally from less supervision. To encourage the development of image captioning models that can learn visual concepts from alternative data sources, such as object detection datasets, we present the first large-scale benchmark for this task. Dubbed ‘nocaps’, for novel object captioning at scale, our benchmark consists of 166,100 human-generated captions describing 15,100 images from the Open Images validation and test sets. The associated training data consists of COCO image-caption pairs, plus Open Images image-level labels and object bounding boxes. Since Open Images contains many more classes than COCO, nearly 400 object classes seen in test images have no or very few associated training captions (hence, nocaps). We extend existing novel object captioning models to establish strong baselines for this benchmark and provide analysis to guide future work.
- Open Access
nocaps: novel object captioning at scale
Image captioning models have achieved impressive results on datasets containing limited visual concepts and large amounts of paired image-caption training data. However, if these models are to ever function in the wild, a much larger variety of visual concepts must be learned, ideally from less supervision. To encourage the development of image captioning models that can learn visual concepts from alternative data sources, such as object detection datasets, we present the first large-scale benchmark for this task. Dubbed 'nocaps', for novel object captioning at scale, our benchmark consists of 166,100 human-generated captions describing 15,100 images from the OpenImages validation and test sets. The associated training data consists of COCO image-caption pairs, plus OpenImages image-level labels and object bounding boxes. Since OpenImages contains many more classes than COCO, nearly 400 object classes seen in test images have no or very few associated training captions (hence, nocaps). We extend existing novel object captioning models to establish strong baselines for this benchmark and provide analysis to guide future work on this task.
Open Accessnocaps: novel object captioning at scale
Image captioning models have achieved impressive results on datasets containing limited visual concepts and large amounts of paired image-caption training data. However, if these models are to ever function in the wild, a much larger variety of visual concepts must be learned, ideally from less supervision. To encourage the development of image captioning models that can learn visual concepts from alternative data sources, such as object detection datasets, we present the first large-scale benchmark for this task. Dubbed 'nocaps', for novel object captioning at scale, our benchmark consists of 166,100 human-generated captions describing 15,100 images from the OpenImages validation and test sets. The associated training data consists of COCO image-caption pairs, plus OpenImages image-level labels and object bounding boxes. Since OpenImages contains many more classes than COCO, nearly 400 object classes seen in test images have no or very few associated training captions (hence, nocaps). We extend existing novel object captioning models to establish strong baselines for this benchmark and provide analysis to guide future work on this task.
- 1 April 2019
- Published by Springer Nature in Behavior Research Methods
- Vol. 52 (1) , 264-278
WordSeg: Standardizing unsupervised word form segmentation from text
A basic task in first language acquisition likely involves discovering the boundaries between words or morphemes in input where these basic units are not overtly segmented. A number of unsupervised...
- 1 April 2019
- Published by Springer Nature in Behavior Research Methods
- Vol. 52 (1) , 264-278
WordSeg: Standardizing unsupervised word form segmentation from text
A basic task in first language acquisition likely involves discovering the boundaries between words or morphemes in input where these basic units are not overtly segmented. A number of unsupervised...