{"id":31583,"date":"2022-10-19T02:37:00","date_gmt":"2022-10-19T02:37:00","guid":{"rendered":"https:\/\/www.lifeandnews.com\/articles\/?p=31583"},"modified":"2022-10-20T20:42:40","modified_gmt":"2022-10-20T20:42:40","slug":"ai-is-changing-scientists-understanding-of-language-learning-and-raising-questions-about-an-innate-grammar","status":"publish","type":"post","link":"https:\/\/www.lifeandnews.com\/articles\/ai-is-changing-scientists-understanding-of-language-learning-and-raising-questions-about-an-innate-grammar\/","title":{"rendered":"AI is changing scientists\u2019 understanding of language learning \u2013 and raising questions about an innate\u00a0grammar"},"content":{"rendered":"\n

Morten H. Christiansen<\/a>, Cornell University<\/a><\/em> and Pablo Contreras Kallens<\/a>, Cornell University<\/a><\/em><\/p>\n\n\n\n

Unlike the carefully scripted dialogue found in most books and movies, the language of everyday interaction tends to be messy and incomplete, full of false starts, interruptions and people talking over each other. From casual conversations between friends, to bickering between siblings, to formal discussions in a boardroom, authentic conversation<\/a> is chaotic. It seems miraculous that anyone can learn language at all given the haphazard nature of the linguistic experience.<\/p>\n\n\n\n

For this reason, many language scientists \u2013 including Noam Chomsky<\/a>, a founder of modern linguistics \u2013 believe that language learners require a kind of glue to rein in the unruly nature of everyday language. And that glue is grammar: a system of rules for generating grammatical sentences.<\/p>\n\n\n\n

Children must have a grammar template wired into their brains<\/a> to help them overcome the limitations of their language experience \u2013 or so the thinking goes.<\/p>\n\n\n\n

This template, for example, might contain a \u201csuper-rule\u201d that dictates how new pieces are added to existing phrases. Children then only need to learn whether their native language is one, like English, where the verb goes before the object (as in \u201cI eat sushi\u201d), or one like Japanese, where the verb goes after the object (in Japanese, the same sentence is structured as \u201cI sushi eat\u201d).<\/p>\n\n\n\n

But new insights into language learning are coming from an unlikely source: artificial intelligence. A new breed of large AI language models can write newspaper articles<\/a>, poetry<\/a> and computer code<\/a> and answer questions truthfully<\/a> after being exposed to vast amounts of language input. And even more astonishingly, they all do it without the help of grammar.<\/p>\n\n\n\n

Grammatical language without a grammar<\/h2>\n\n\n\n

Even if their choice of words is sometimes strange<\/a>, nonsensical<\/a> or contains racist, sexist and other harmful biases<\/a>, one thing is very clear: the overwhelming majority of the output of these AI language models is grammatically correct. And yet, there are no grammar templates or rules hardwired into them \u2013 they rely on linguistic experience alone, messy as it may be.<\/p>\n\n\n\n

GPT-3, arguably the most well-known of these models<\/a>, is a gigantic deep-learning neural network<\/a> with 175 billion parameters. It was trained to predict the next word in a sentence given what came before across hundreds of billions of words from the internet, books and Wikipedia. When it made a wrong prediction, its parameters were adjusted using an automatic learning algorithm.<\/p>\n\n\n\n

Remarkably, GPT-3 can generate believable text reacting to prompts such as \u201cA summary of the last \u2018Fast and Furious\u2019 movie is\u2026\u201d or \u201cWrite a poem in the style of Emily Dickinson.\u201d Moreover, GPT-3 can respond<\/a> to SAT level analogies, reading comprehension questions and even solve simple arithmetic problems \u2013 all from learning how to predict the next word.<\/p>\n\n\n\n

\"artist's<\/a>
An AI model and a human brain may generate the same language, but are they doing it the same way? Just_Super\/E+ via Getty Images<\/a><\/figcaption><\/figure>\n\n\n\n

Comparing AI models and human brains<\/h2>\n\n\n\n

The similarity with human language doesn\u2019t stop here, however. Research published in Nature Neuroscience demonstrated that these artificial deep-learning networks seem to use the same computational principles as the human brain<\/a>. The research group, led by neuroscientist Uri Hasson<\/a>, first compared how well GPT-2<\/a> \u2013 a \u201clittle brother\u201d of GPT-3 \u2013 and humans could predict the next word in a story taken from the podcast \u201cThis American Life\u201d: people and the AI predicted the exact same word nearly 50% of the time.<\/p>\n\n\n\n

The researchers recorded volunteers\u2019 brain activity while listening to the story. The best explanation for the patterns of activation they observed was that people\u2019s brains \u2013 like GPT-2 \u2013 were not just using the preceding one or two words when making predictions but relied on the accumulated context of up to 100 previous words. Altogether, the authors conclude: \u201cOur finding of spontaneous predictive neural signals as participants listen to natural speech suggests that active prediction may underlie humans\u2019 lifelong language learning<\/a>.\u201d<\/p>\n\n\n\n

A possible concern is that these new AI language models are fed a lot of input: GPT-3 was trained on linguistic experience equivalent to 20,000 human years<\/a>. But a preliminary study<\/a> that has not yet been peer-reviewed found that GPT-2 can still model human next-word predictions and brain activations even when trained on just 100 million words. That\u2019s well within the amount of linguistic input that an average child might hear during the first 10 years of life<\/a>.<\/p>\n\n\n\n

We are not suggesting that GPT-3 or GPT-2 learn language exactly like children do. Indeed, these AI models do not appear to comprehend much<\/a>, if anything, of what they are saying, whereas understanding is fundamental to human language use<\/a>. Still, what these models prove is that a learner \u2013 albeit a silicon one \u2013 can learn language well enough from mere exposure to produce perfectly good grammatical sentences and do so in a way that resembles human brain processing.<\/p>\n\n\n\n

\"little<\/a>
More back and forth yields more language learning. Westend61 via Getty Images<\/a><\/figcaption><\/figure>\n\n\n\n

Rethinking language learning<\/h2>\n\n\n\n

For years, many linguists have believed<\/a> that learning language is impossible without a built-in grammar template. The new AI models prove otherwise. They demonstrate that the ability to produce grammatical language can be learned from linguistic experience alone. Likewise, we suggest that children do not need an innate grammar<\/a> to learn language.<\/p>\n\n\n\n

\u201cChildren should be seen, not heard\u201d goes the old saying, but the latest AI language models suggest that nothing could be further from the truth. Instead, children need to be engaged in the back-and-forth of conversation<\/a> as much as possible to help them develop their language skills. Linguistic experience \u2013 not grammar \u2013 is key to becoming a competent language user.<\/p>\n\n\n\n

Morten H. Christiansen<\/a>, Professor of Psychology, Cornell University<\/a><\/em> and Pablo Contreras Kallens<\/a>, Ph.D. Student in Psychology, Cornell University<\/a><\/em><\/p>\n\n\n\n

This article is republished from The Conversation<\/a> under a Creative Commons license. Read the original article<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

Morten H. Christiansen, Cornell University and Pablo Contreras Kallens, Cornell University Unlike the carefully scripted dialogue found in most books and movies, the language of everyday interaction tends to be messy and incomplete, full of false starts, interruptions and people talking over each other. From casual conversations between friends, to bickering between siblings, to formal […]<\/p>\n","protected":false},"author":44,"featured_media":31584,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[292,3410],"tags":[10656,9646,12771,9290,3173,6889,2755],"_links":{"self":[{"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/posts\/31583"}],"collection":[{"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/users\/44"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/comments?post=31583"}],"version-history":[{"count":2,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/posts\/31583\/revisions"}],"predecessor-version":[{"id":31590,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/posts\/31583\/revisions\/31590"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/media\/31584"}],"wp:attachment":[{"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/media?parent=31583"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/categories?post=31583"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.lifeandnews.com\/articles\/wp-json\/wp\/v2\/tags?post=31583"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}