In-Context Learning

What is in-context learning?

You don't retrain the model — you drop a few examples into the prompt and it figures out the task on the spot. In-context learning (ICL) does exactly that: not a single weight changes, yet the model picks up a new task from the information sitting in its context window.

How does it work?

Examples, not rules
Instead of writing a paragraph explaining "translate English to Chinese", you just show two or three "input → output" pairs and the model follows along. That's few-shot prompting.
Attention lands on the examples
As the model generates its answer, it focuses on those examples, "reads" the pattern of the task, and applies it to your new question.

Why does it matter?

New tasks at zero cost
The old way of getting a model to do something new was fine-tuning: data, compute, time. ICL needs none of that — you just edit the prompt.
Switch tasks on a whim
Want a different task? Swap in a different set of examples. That flexibility is a big reason prompt engineering took off.

Where does it fall short?

ICL is bounded by the context window — too many examples overflow it and start stepping on each other. It also doesn't "stick" the way fine-tuning does: you have to re-supply the examples every time. But for most everyday tasks, it's more than good enough.

Bottom line: in-context learning teaches an AI by example — no rules, no training, just a few demonstrations.