--
You received this message because you are subscribed to the Google Groups "Power Satellite Economics" group.
To unsubscribe from this group and stop receiving emails from it, send an email to power-satellite-ec...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/power-satellite-economics/CAPiwVB7MuedpRiiE4ZSzgOSK%2BY1PKf2GnHYKLEmkZhJD8LsLpA%40mail.gmail.com.
To view this discussion visit https://groups.google.com/d/msgid/power-satellite-economics/005701db8baf%24d9d627b0%248d827710%24%40sympatico.ca.
Erinn;
Rather off-topic from space power economics, but maybe worthwhile, to scotch a particularly pernicious current misunderstanding:
You wrote:
- Explore different ways of asking it to double--check itself, evaluate its own logic, search for flaws in its work, and to show both its methodology and its sources.
The following is based on what I’ve gleaned from some reading into LLMs. If any of this is wrong, I’d appreciate pointers as to what’s wrong, and what the right answer is. With that caveat, I’ll (like ChatGPT 😊) proceed to pontificate as if I was full of certainty…
You’re succumbing to the delusion that these LLMs “think.” They don’t. They simply regurgitate text patterns from the corpus of works on which they were trained (some call them “spicy auto-complete”) --- with some impressive bells and whistles added, true (enabling them to go beyond text, to images and audio, among other things).
If there exists in their training corpus some text that matches the answer you’re looking for, then it may regurgitate that. Or, it may come up with some other amalgamation of other text strings. But there’s no “thinking” or “self-awareness” involved in doing this. It *cannot* “evaluate its own logic,” because it cannot evaluate anything. It doesn’t do “evaluation,” it does text-string generation based on semi-random plunges into the probability matrix that was generated when its training corpus was digested. (Experiment: ask it the same question multiple times, you’ll get multiple contradictory answers, that comes from the random-plunge part.)
There’s a classic textbook on information theory that I first read many decades ago (my dad studied from it when he went to grad school):
that was first written in 1961; it compiles and explains the foundational concepts of information theory that Shannon and others developed not too long before then. (What an excellent book! I highly recommend it.) Several of its concepts have stuck in my mind for decades. One of those is his excellent explanation (in chapter 3) of Shannon’s demonstration of how to put together zero-order, first-order, second-order etc. approximations to strings of characters, based on such “random plunges” into a codified “training corpus”, such that (as you go to higher and higher orders) first strings of words begin to appear, then with even higher orders those strings seem to begin to make sense. The “training” is actually statistical analysis of every text substring in a “training” corpus of text (e.g., the contents of lots of books), to see how many times a given string of length n appears (for an order-n approximation); once that’s been done for all length-n strings, then the probability of finding any one of those strings, if you start sampling the corpus at any given point, is known. The text-generation bit just uses a random number generator, weighted by those probabilities, to select which string of text from the corpus to parrot back.
AFAIK, ChatGPT etc. are modern updated versions of that basic approach. One innovation they use is in the statistical analysis side. Using longer text strings (i.e., a higher value of n) produces results that seem more and more “sensible.” But, there’s a combinatorial explosion that happens as you go to larger values of n; before too long, the amount of computation needed to do the statistical analysis (if done via brute-force) becomes impossibly large. From my reading into LLMs, they *do* use a particular AI-related technique (computational neural networks) to do a much quicker, approximate analysis of the training corpus, to come up with the (enormous) probability table. (I believe there are other innovations as well, such as algorithms (not AI) that enforce grammar rules.)
So, my current understanding is that the only place where an AI technique is used in thee LLMs, is the use of neural networks in the analysis of the training corpus, to produce the probability table. And that the actual software that you interact with when you use ChatGPT/etc., is purely algorithmic (not AI), basically selecting and stitching together randomly-selected strings of text using a random-number generator, weighted by the probabilities in that table.
- Kieran