The scaling law only states that more resources yield lower training loss (https://en.wikipedia.org/wiki/Neural_scaling_law). So for an LLM I guess training loss means its ability to predict the next token.
So maybe the real question is: is next token prediction all you need for intelligence?
As a human, I oftentimes can solidify ideas by writing them out and editing my writing in a way that wouldn’t really work if I could only speak them aloud a word at a time, in order.
And before we go to “the token predictor could compensate for that…” maybe we should consider that the reason this is the case is because intelligence isn’t actually something that can be modeled with strings/tokens.
Is there a reason that reflected light off a vertical plane has a particular polarization? I know that light reflected off the ground gets polarized (which is why polarized sunglasses help so much) but that reflection is at a steep angle and not near/at 90 degrees.
https://farside.ph.utexas.edu/teaching/em/lectures/node104.h... has a detailed derivation of dielectric reflection, but you can also skip it and just look at figure 57 at the bottom showing the predicted reflectances for the two directions of polarization depending on the incidence angle.
You're right that for perfectly vertical reflection, the polarization doesn't matter, but you're unlikely to exactly hit that. For angles between 0 and 90 degrees, light polarized parallel to the surface is always reflected better. If you perfectly hit Brewster's angle https://en.wikipedia.org/wiki/Brewster%27s_angle the light will be completely linearly polarized, but that is equally unlikely. So in general you're going to get mixed polarization that's slightly biased in one direction.
Thanks! Fun fact, do you know that apparently in reflections off a mirror, the original photon is destroyed and replaced by an "identical" one apparently?
Non-zero probability I think, one interesting measure of overfitting I've seen is contamination (where the model has seen the exact questions it is being evaluated on) see stats at https://hitz-zentroa.github.io/lm-contamination/
> In 2021 and the first half of 2022 (when most of the 2022 activity happened) we essentially crammed 5 years worth of funding into an 18 month period. Series Bs and Cs were raised when companies were at the typical Series A milestones. Normal round sizes doubled or tripled. Every type of investor was broadly operating in a “risk on” mindset given the ZIRP environment, and the venture capital ecosystem was no exception.
