Findings suggest that scaling the parameters of the model consistently yields performance improvements as many other previous works have reported. On the other hand, the authors find that adding examples to improve performance depends more on the task’s format. An interesting finding in the paper is that in some tasks like extractive question answering and classification benefit a lot from additional examples. In some cases, collecting a few hundred examples is “worth” billions of parameters. A possible explanation for these findings is that problems like open question answering require more recalling of specific information while other tasks with restricted output space transfer across examples and can be learned with small amounts of labeled data. — Updated on 2021-10-17 14:42:13 — Group: #Public