Over the last two years, artificial neural network models have come close to (and in many cases surpassed) human-level performance on most preexisting benchmarks for language understanding. While many of these benchmarks have known limitations, these models are nonetheless strikingly effective, and it is increasingly plausible that they acquire substantial knowledge of the structure of English during a training procedure that relies almost exclusively on raw unannotated text. 

This talk surveys an ongoing line of research that attempts to use acceptability judgments as a lens through which to understand what these models are learning, and presents initial results that suggest that it is possible to learn to produce human-like patterns of acceptability judgments from raw text alone. In particular, I will briefly survey the striking results that the field has seen with large-scale neural network language models like ELMo, GPT-2, and BERT; and then discuss experiments with the CoLA corpus of acceptability judgments from published Linguistics literature and the BLiMP corpus of expert-constructed minimal pairs.

About the speaker:
Sam Bowman has been on the faculty at NYU since 2016, when he completed a PhD with Chris Manning and Chris Potts at Stanford. At NYU, Sam is jointly appointed between the Department of Linguistics and the new school-level Center for Data Science, which focuses on machine learning, and is also a co-PI of the CILVR machine learning lab and an affiliate member of the Courant Institute's Department of Computer Science. Sam's research focuses on data, evaluation techniques, and modeling techniques for sentence and paragraph understanding in natural language processing and on applications of machine learning to scientific questions in linguistic syntax and semantics. Sam organized a twenty-three person research team at JSALT 2018 and received a 2015 EMNLP Best Resource Paper Award, a 2017 Google Faculty Research Award, and a 2019 *SEM best paper award.

---

 

To join the remote presentation: