How ChatGPT Understands Your Questions?
BSc CSIT Graduate
Back in the days we used to ask questions to teachers, friends or we used to google it. But nowadays we don't do that anymore. Today, many of us simply open ChatGPT and ask our question in natural language.
But have you ever wondered how chatGPT understand everything we say and gives us answers?
Today, in this blog I will try to explain how chatGPT understand questions and give us answers. The topics will be covered are:
By the end of this article, you'll understand what happens inside an LLM from the moment you type a prompt until ChatGPT generates its response.
What is an LLM?
What does LLM stand for?
LLM stands for Large Language Model.
Large: The model is trained on enormous amount of text data. It has been trained on most of the available data on internet ( like books, blogs, websites, code etc).
Language: It is called a Language Model because it is designed to understand and generate human language.
Model: In machine learning, a model is a mathematical system that learns patterns from data. It isn't a database of facts—it learns statistical relationships between words and tokens.
So, the LLM is a machine learning model that has been trained using large amount of data to make it able to communicate in human language by predicting next token
What problems the LLMs solve?
Before LLMs, humans often had to communicate with computers using commands or programming languages. LLMs solved that and Language related problems like answering quetions, text generation, writing and debugging code, translation, summarization.
Popular examples of LLM
Popular LLMs are: GPT (OpenAI), Claude (Anthropic), Gemini (Google), Llama (Meta), Mistral, Qwen, DeepSeek, and Command (Cohere)
What Happens When You Send a Message to ChatGPT?
When you send a message to chatGPT, first it breaks prompt into tokens(we will talk later about it) and converts the prompt into tokens and numerical representations so that LLM (GPT) could understand that and do processing and give us what we want.
After converting prompt to it's understandable form GPT predicts one token at a time append that token and again generate next token and keep predicting until it generates whole response. The whole response is not generated at once but one by one token.
Why responses are not copied from the internet?
LLMs are not something like the databases to retrieve instead they generate. LLMs generate reponses on the go based the pattern they have learned from the training data. LLMs are not connected to the live, and does not store any documents they predict next token based on the training data. (That's why they are called GPT: generative pre-trained transformer).
Why Computers Don't Understand Human Language
As we know computers process mathematical logic through binary code (1s and 0s). But human language is messy and contextual . So it is hard for computer to understand human language.
Are you sure?
Are you sure? 😡
Are you sure?😂
Are you sure? 😉
Are you sure? 😢
Computers don’t understand emojis as emotions — they only see them as different Unicode numbers. But we as a human can clearly see the difference.
The main thing that makes computers bad at understanding human language is Context and Intent.
To make computers to understand we need to covert natural language into number. For that thing we use concept called Tokennization.
Tokenization
When we give a prompt to ChatGPT the first step is tokenization. Our input text is broken into chunks called tokens. Different LLMs may use different tokenizers and vocabularies.
Each token has a unique number associated with it called a token ID. If we assign a number to every single character that would be inefficient. A token is a sequence of characters, not necessarily a complete word.
A word is what humans read and write, while a token is the unit an LLM processes. A token is not always a complete word—it can be a whole word, part of a word, punctuation, a number, or an emoji. The exact tokenization depends on the model's tokenizer.
For example:
"Write a poem about Nepal."
There are just 5 words. But Using the GPT-4/GPT-3.5 tokenizer, this sentence is split into 6 tokens as shown below.
And Token IDs are like : [8144, 264, 33894, 922, 50064, 13].
The process of converting tokens into corresponding numbers is called Encoding.
Every model has its own vocabulary, which is a dictionary that maps tokens to their corresponding token IDs. So our text is broken into tokens based on that dictionary so it can easily be converted into numbers for further processing.
Transformer
Before 2017, many AI language models struggled to understand long sentences because they processed words one by one. This made it difficult to remember information from earlier parts of a sentence.
A transformer is the core architecture of modern Large Language Models (LLMs). Its job is to understand the relationships between all the tokens in the input and use that understanding to predict the next token.
Everything changed with the publication of Attention Is All You Need, a research paper that introduced the Transformer architecture. Today, almost every modern Large Language Model (LLM), including GPT, Gemini, Claude, and Llama, is built on this architecture.
A Transformer is called a Transformer because it transforms simple token representations into rich, context-aware representations. Instead of treating every word independently, it learns the meaning of each word based on the words around it.
Embeddings
After tokenization and encoding, Now LLM has the token ID's like: [8144, 264, 33894, 922, 50064, 13]. These numbers themself don't have real world meaning.
This is where the Embeddings comes into picture.
An embedding is a high-dimensional vector (we use 3d for visualization) that represent a meaning of token. Instead of treating words as plain text, the model places them in a mathematical space where similar concepts are located close to one another and form clusters.
Embeddings place similar concepts near each other, and similar semantic relationships are represented by similar vectors.
Embeddings capture two important properties:
a.semantics similarities and
b. semantic relationships
For example:
Countries like Nepal, India , USA, France naturally form a cluster.
Similary pet animals like cat , dog are close to eachother than a truck .
Words that have similar meanings or play similar roles tend to be close together in this embedding space.
As shown in the diagram above, embeddings don't just group similar words together—they also learn relationships between them.
For example, Nepal → Kathmandu represents the same relationship as India → New Delhi
Mathematically, we can write:
Kathmandu − Nepal ≈ New Delhi − India
This doesn't mean the values are exactly equal. The symbol ≈ means approximately equal. It shows that both vectors represent a similar semantic relationship:
Country → Capital City
Similarly, moving from Nepal to India is somewhat similar to moving from Kathmandu to New Delhi because the concepts are related in a similar way.
Embeddings allow the model to understand both similarity and relationships, which is one of the reasons modern LLMs perform so well.
Positional Encoding
Embeddings tell the model what a word means, but they don't tell it where the word appears in the sentence.
Consider following two sentences:
Bipin loves coffee.
Coffee loves Bipin.
Both sentences contain exactly the same words. Without knowing the order of those words, the model would struggle to understand that the meanings are completely different.
This is where Positional Encoding comes in. Positional encoding adds information about the position of every token before it enters the Transformer. This helps the model understand not only the meaning of words but also their order.
Self Attention
One of the biggest innovations introduced by Transformers is the Self-Attention mechanism.
Some words have multiple meanings depending on the context.
NIC Bank
River Bank
Both sentences contain the word Bank, but the meaning is completely different.
How does the model know which meaning to choose? Ans: Self Attention
Self-attention allows every token to "look at" every other token in the sentence before deciding its meaning. In the first sentence , the word bank pays attention to NIC , similarly in second sentence Bank pays attention to River and the model can determine the correct meaning of the word based on its context.
Instead of processing words independently, every token exchanges information with the other tokens in the sentence. This allows the model to build a much richer understanding of the context.
This is why Transformers are so good at understanding language.
Multi-head Attention
Multi-Head Attention simply means that the Transformer performs this attention process multiple times in parallel.
Each attention head can focus on a different type of relationship.
For example, in the sentence:
"Bipin gave his book to Ram because he had finished reading it."
One attention head might learn:
- Who he refers to.
Another head might learn:
- What it refers to.
Another might focus on:
- The grammatical structure.
The outputs from all these attention heads are then combined, giving the model a richer understanding of the sentence than a single attention mechanism could provide.
Feed Forward Network (FFN)
After the attention mechanism gathers information from other tokens, the result passes through a Feed Forward Neural Network (FFN).
You can think of the FFN as another processing step that refines the information learned by self-attention.
While self-attention helps a token understand its relationship with other tokens, the Feed Forward Network helps the model learn more complex patterns before passing the information to the next Transformer layer.
This process is repeated many times across multiple Transformer layers, gradually building a richer understanding of the input.
Logits
After the input passes through all the Transformer layers, the model still hasn't chosen the next token.
Instead, it produces a raw score for every token in its vocabulary.
These scores are called logits.
For example, suppose the prompt is:
The capital of Nepal is
The model might produce something like:
| Token | Logit |
|---|---|
| Kathmandu | 12.4 |
| Pokhara | 6.8 |
| India | 2.1 |
| Banana | -3.7 |
These numbers are not probabilities.
They simply represent how strongly the model believes each token could be the next one.
Temperature
Temperature controls how confident or random the model becomes. Instead of changing logits themselves it scales them
newLogit = logit/temperature
Where,
T = 1 → no change
T < 1 → more confident
T > 1 → more random
With a low temperature (for example 0.2), the model almost always chooses the highest-probability token.
This makes the response more focused and predictable.
With a high temperature (for example 1.2), the probabilities become more balanced, making it more likely that less probable tokens are selected.
As a result, responses become more creative, but they may also become less accurate.
Softmax
After temperature is applied we get the numbers but they are still not probabilities . The new logits are passed through a mathematical function called Softmax. It makes everything sum to exactly 1.
Now we finally have a probability distribution.
A probability distribution is simply
the probability assigned to every possible next token.
Once the probability distribution is calculated, the model selects the next token. In many cases, it chooses the most probable token, although depending on the decoding strategy and temperature, it may also sample from several likely tokens. The selected token is appended to the input, and the entire process repeats until the response is complete.
Putting Everything Together
Let's see the complete journey of your prompt.
Suppose you ask ChatGPT:
Write a poem about Nepal.
The following steps happen:
The text is broken into tokens.
Tokens are converted into Token IDs (Encoding).
Token IDs become Embeddings.
Positional Encoding is added.
The Transformer processes the tokens using Multi-Head Attention and Feed Forward Networks across multiple layers.
The model generates Logits for every possible token.
Temperature adjusts these logits.
Softmax converts them into probabilities.
The next token is selected.
That token is appended to the sentence.
The process repeats until the response is complete.
Context Window
An LLM doesn't only look at the last word you typed. It also considers a limited amount of previous conversation called the context window.
For example:
User:
My name is Bipin.
Later:
What's my name?
ChatGPT can answer Bipin because your earlier message is still inside its context window.
If the conversation becomes too long and earlier messages fall outside the context window, the model may no longer remember them.
Training vs Inference
Everything we've discussed so far happens during Inference, which is the phase we use when chatting with ChatGPT.
However, before the model can answer questions, it must first go through a Training Phase.
During training, the model is shown billions of examples of text.
Suppose the training sentence is:
<BOS> Bipin is handsome <EOS>
Here:
BOS means Beginning of Sequence.
EOS means End of Sequence.
Initially, the model makes random predictions.
Instead of predicting handsome, it might predict something completely different.
The correct next token is called the label.
The model compares its prediction with the correct label using a mathematical function called Cross Entropy Loss, which measures how wrong the prediction is.
The larger the loss, the worse the prediction.
Using this loss, the model updates its internal parameters through backpropagation, allowing it to make better predictions in the future.
This process is repeated billions of times until the model becomes good at predicting the next token.
Inference Phase
Once training is complete, the model enters the Inference Phase.
This is the phase we use every day when interacting with ChatGPT.
During inference:
No labels are provided.
No Cross Entropy Loss is calculated.
No backpropagation takes place.
The model simply uses everything it learned during training to predict the next token.
Conclusion
Although ChatGPT may seem like magic, it follows a sequence of well-defined steps. Your prompt is first tokenized, converted into numerical representations, processed through multiple Transformer layers, and finally used to predict one token at a time until a complete response is generated.
The next time you ask ChatGPT a question, you'll know that it isn't searching a database for the answer—it is generating one token at a time using patterns learned during training.
