RLHF

What is RLHF?

Right after pretraining, a large model is like a brilliant kid who hasn't learned to filter: it knows a lot, but it can't tell what's appropriate to say, and it doesn't really know which answer you're hoping for. RLHF — reinforcement learning from human feedback — is the process of giving it a mentor. You use human preferences to slowly shape it into something that's both capable and tactful.

Why does it need humans at all?

Fluency isn't the same as correctness
A model can generate paragraphs of text that aren't what you wanted. Ask it how to lose weight and it might hand you a list of dangerous crash diets. The value of human feedback is teaching it: that answer is bad, this one is good.
"Good" has no fixed rule — people have to teach it
What counts as helpful, harmless, or on-target is deeply subjective. A machine can't learn that on its own; it needs a mountain of human labels.

How RLHF works, roughly

Step one: train a reward model
Have the model produce several answers to the same question, then ask humans to rank them from best to worst. That ranking data trains a "scorer" that learns to grade answers — the higher the score, the more humans liked it.
Step two: use the scorer to reshape the model
Once you have a scorer, you let the model generate answers, get them scored, and nudge the model toward higher scores. Rinse and repeat, and the model gradually gets better at reading what people want.

What problem does it actually solve?

RLHF is a big part of why products like ChatGPT feel useful. It makes answers land closer to human values, hallucinate less, and more readily admit "I don't know." Without RLHF, a large model is merely articulate. With it, the model learns to be genuinely helpful — and to take feedback.

Bottom line: RLHF uses human judgment as a ruler to shape AI, turning it from something that can talk into something that knows the right thing to say.