The results exceeded the researchers' expectations. The team attributes them directly to data quality, as the title of the paper suggests: "Textbooks is all you need," in reference to Google's research on the Transformer breakthrough ("Attention is all you need").
If I have the right context, they selected HIGHLY QUALIFIED inputs. Which probably meant their training set was good as a seed to another system because it had been pre-groomed.
The results exceeded the researchers' expectations. The team attributes them directly to data quality, as the title of the paper suggests: "Textbooks is all you need," in reference to Google's research on the Transformer breakthrough ("Attention is all you need").
If I have the right context, they selected HIGHLY QUALIFIED inputs. Which probably meant their training set was good as a seed to another system because it had been pre-groomed.
But, it's still impressive.