How Do Machines ‘Grok’ Data?

[John Clark]

When training a neural network programmers are always on the lookout for something called "overfitting" when the AI seems to stop generalizing and just memorizes the training data; typically that's the point where the training stops. However, when a researcher at OpenAI was working with a small neural network and clear signs of overfitting occurred, he forgot to turn it off and then went on vacation. When he got back from vacation,  he found to his surprise that the network had very dramatically improved its performance. It's like it had obtained a profound understanding of the  data, he called it "Grokking''. I've had the experience, and you probably have too, of studying something for a long time and not understanding it, and then suddenly bang there is an aha moment and everything becomes clear. It sounds sort of like that.

How Do Machines ‘Grok’ Data?

 John K Clark    See what's on my new list at  Extropolis

spg

[Brent Meeker]

What I said.

https://www.patreon.com/posts/worst-prediction-102409950

Brent

[John Clark]

On Wed, Apr 17, 2024 at 12:07 AM Brent Meeker <meeke...@gmail.com> wrote:

What I said.

https://www.patreon.com/posts/worst-prediction-102409950

This is a good example of why I'm not a big fan of Sabine Hossenfelder, she's always right and everybody else is always wrong. In 1967 Yakov Zel’dovich was the first to formally calculate that value of the cosmological constant based on quantum theory, as we currently understand it, produces a value that was obviously 120 orders of magnitude too high.  Zel’dovich was not stupid enough to claim that's what the value actually was, he was just pointing out that Physics had a serious problem. And she complains that you only get that 10^120 figure if you assume that the Planck length and Planck mass are important, but if you don't make that assumption then you get a value that is infinitely too high. Hossenfelder is a physicist so she knows all this of course, but she loves to be provocative and a contrarian because it's good Clickbait.

John K Clark    See what's on my new list at  Extropolis

hjw

 

[Brent Meeker]

On the contrary I think people who cite this as a great failure of QFT are producing clickbait.  No physicist has ever taken the number seriously.  I think it was always a misbegotten number because it's really calculating the eigenvalue of the energy in a Planck size cube.  But the eigenvalue of the energy is what you would get if you measured the vacuum energy in that cube. If you really tried to make such a measurement your instrument would have to supply the energy to define the Planck cube.  Absent a measurment, there's no reason to suppose the vacuum has all that energy.

Brent

--
You received this message because you are subscribed to the Google Groups "Everything List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to everything-li...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/everything-list/CAJPayv1F-XD%2B-rB3MCj6HOtyOFPsWCn4FZ4P%3DTexYUhkLPA2fw%40mail.gmail.com.

[John Clark]

On Wed, Apr 17, 2024 at 2:39 PM Brent Meeker <meeke...@gmail.com> wrote:

On the contrary I think people who cite this as a great failure of QFT are producing clickbait.  No physicist has ever taken the number seriously. 

Well of course no physicist has ever taken that number seriously, that is the point!!  The calculated number is not just wrong, it is ridiculously wrong.  Everybody agrees it is wrong but there is no agreement about exactly, or even approximately, why it is wrong.  

  John K Clark    See what's on my new list at  Extropolis

wiw