Editorial: Perspectives for natural language processing between AI, linguistics and cognitive science

Abstract

Natural Language Processing (NLP) today-like most of Artificial Intelligence (AI)-is much more of an “engineering” discipline than it originally was, when it sought to develop a general theory of human language understanding that not only translates into language technology, but that is also linguistically meaningful and cognitively plausible. At first glance, this trend seems to be connected to the rapid development in the last 10 years that was driven to a large extent by the adoption of deep learning techniques. However, it can be argued that the move toward deep learning has the potential of bringing NLP back to its roots after all. Some recent activities and findings in this direction include: Techniques like multi-task learning have been used to integrate cognitive data as supervision in NLP tasks (Barrett et al., [1]); Pre-training/fine-tuning regimens are potentially interpretable in terms of cognitive mechanisms like general competencies applied to specific tasks (Flesch et al., [3]); The ability of modern models for ‘few-shot' or even ‘zero-shot' performance on novel tasks mirrors human performance (Srivastava et al., [7]); Evidence of unsupervised structure learning in current neural network architectures that mirrors classical linguistic structures (Hewitt and Manning, [4]; Tenney et al., [8]). In terms of developing systems endowed with natural language capabilities, the last generation of neural network architectures has allowed AI and NLP to make unprecedented progress. Such systems (e.g., the GPT family) are typically trained with huge computational infrastructures on large amounts of textual data from which they acquire knowledge thanks to their extraordinary ability to record and generalize the statistical patterns found in data. However, the debate about the human-like semantic abilities that such “juggernaut models” really acquire is still wide open. In fact, despite the figures typically reported to show the success of AI on various benchmarks, other research argues that their semantic competence is still very brittle (Lake and Baroni, [5]; Bender and Koller, [2]; Ravichander et al., [6]).