Companies are optimizing for all the big benchmarks. This is why there is so little correlation between benchmark performance and real world performance now.
Yes, LLMs have become extremely good at coding (not software engineer though). But try using them for anything original that cannot be adapted from GitHub and Stack Overflow. I haven't seen much improvement at all at such tasks.
No shot, their classic engineering ability has exploded too.
The amount of information available online about optics is probably <0.001% of what is available for software, and they can just breeze through modeling solutions. A year ago was immediate face-planting.
The gains are likely coming from exactly where they say they are coming from - scaling compute.
