AI physics simulation (weather, climate, aerodynamics) requires massive synthetic data generation; 200 terabytes of simulation data compresses to 200 MB model checkpoint, enabling million-times compression

“if you run those simulations with more than a billion degrees of freedom, it generates around 200 terabytes of data. Now what's amazing is the way that the modern architectures work... when you take them through these transformers and use some very clever techniques to tokenize this information, you find that the model checkpoint, and this isn't a typo on the slide, is only around 200 megabytes. It is more than 10,000 times faster to do the prediction”