👋 Surely everyone has heard of ChatGPT - an artificial intelligence that made a big impression on the front pages of newspapers 🗞️. Some people are fascinated by AI 😍, while others are afraid of it 😱, but only a few truly understand what it really is 🤔. The technology that ChatGPT is based on has been known since 2017, when Google published its groundbreaking research paper presenting a new approach to creating Language Models 🤖. But what exactly is a Language Model?
A Language Model is a model that can predict the next word based on previously written words. 📝 This is something that researchers have known since the 1950s! However, in the 80s and 90s, language models underwent a renaissance thanks to the use of modern neural networks 💥, and currently they are developing at an express pace.
Few people realize that language models have been in our pockets for some time now. 📱 Have you noticed while writing a text message that the keyboard suggests the next word for you? That’s the Language Model built into your phone! 🤯 Why does it work? Because every word we use carries certain meanings and associations. Language models approach this issue in a purely mathematical way - based on how often a given word appears in a specific context, they predict what word should appear next.
For example, if you write “Billy put on his backpack and went to…”, most people would predict that the next word is “school” 🏫. Why? Because Billy is associated with a young boy of school age, and a backpack is typical for students. Language models base their predictions on the analysis of many different contexts and words, allowing them to predict the next word with high accuracy.
Of course, the operation of language models is more complicated than what we have presented here in a simplified way, but the basic principle has remained the same for decades. Only the technological possibilities and the amount of data on which models are trained have changed. 💻
If you’re interested in the practical applications of language technology 🤖, it’s worth mentioning the Typly app📱, which is an intelligent writing assistant ✍️. Many of its features utilize our proprietary NLP (Natural Language Processing) engine, which integrates various language models, including ChatGPT 🤯.