But by scoring the model’s sample answers automatically, the training process nudged it bit by bit toward the desired behavior.