In the early stages of training, human contractors play dual roles, acting as both the user and the ideal chatbot. They input these interactions into the model, teaching it to maximize the relevance of words and sentences. Through this, the model learns to generate outputs.
Once the model produces outputs, it undergoes further refinement. Developers step in to train ChatGPT in assigning a reward or ranking. Human trainers rank the outputs from best to worst, and this data gets fed back into the model. This process helps ChatGPT learn to critically evaluate which output is likely to be the best.
However, relying solely on human trainers poses a scalability issue. Human trainers can’t possibly anticipate every potential input and output a user might request. To tackle this, a third step called reinforcement learning is involved. This unsupervised learning method helps the model understand underlying contexts and patterns based on its earlier human-guided training.
