Podcast: How computers manage to deal with language independently

16 May 2023, by Chris Biemann

Powerful language models have become available to the broad world public, making the success of computational linguistics evident: Computer architectures have been developed there that are capable of processing intuitive human language. The behaviour shown by these systems is hardly comprehensible in detail, but the results they deliver are all the more impressive: Assistance systems equipped with these language models can be controlled and generate output as if they actually mastered language themselves – and their potential applications go far beyond the sensational chat bots that dominate public debates. However, GPT and related systems have by no means emerged suddenly. They are the result of a persistent learning process: first attempts to map language into algorithms failed in the mid-twentieth century not only because of a lack of resources or because crucial machine learning methods had not yet been developed, but also because the theories on how to abstractly grasp and systematise the meaning level of human language were insufficient. So what do developers of modern systems do differently from the pioneers of computational linguistics?

In this episode of Digitalgespräch, Chris Biemann provides deep insights into the way modern language models are created and how they work, explaining linguistic theories that come into play. He makes comprehensible why the systems deliver such impressive results and describes what they can be used for in scientific contexts. With hosts Marlene Görger and Petra Gehring, Biemann discusses what resources go into the development of such systems, what happens when language models are trained on nearly the entire internet – and what tasks computational linguists face now that they seem to have achieved their great goal.

Further informationen:
Link to the textbook „Wissensrohstoff Text – eine Einführung in das Text Minig“ von Chris Biemann, Gerhard Heyer und Uwe Quasthoff