Tiny Language Models Come of Age

Using large language models to generate training data for smaller models might be more effective than vacuuming up text from the internet and filtering out the garbage.