SuperCard and hallucinations
Jerzy Dydak
I do all my programming on SuperCard. Here is the weirdest thing that happened to me recently.
You probably heard of ChatGPT being able to hallucinate. Well, my computer was doing it a couple of days ago. Here is a background:
I am lately very tired as I work the whole day on a project with Google DeepMind on testing their AI software on math problems. It is an extension of their work described in the article
https://deepmind.google/discover/blog/alphageometry-an-olympiad-level-ai-system-for-geometry/
My job is to create new and challenging problems in linear algebra, analytic geometry, and algebra. New means they were not published on the internet or in a book. So far I averaged 7 problems a day (I planned 2 or 3) but it has to slow down. I design them the same way as I do for my classes: a problem has to have a decent answer (a whole number in the case of Google DeepMind) and I program them so that I can produce problems students cannot copy answers easily from other students. Sometimes I spend a few hours on a problem if there is a programming mistake or a designing mistake. The strangest thing was that I thought programming was good but direct computing led to wrong answers. What was going on? The first clue was that by clicking a button to get a version with acceptable parameters I was getting zeroes as some of them. That was not part of programming at all. Then, when I was pasting output from my programs to a text file and using it to check computations with WolframAlpha, instead of numbers I was copying some strange text. So it looks that in previous computations copying numbers led to different ones, not what I wanted. Since I am using very big databases of numbers to get the right answers (whole numbers) I decided to scale down databases and everything worked after that. In all my 40 years of using computers I never experienced anything like that. Once every few days programs I design for classes crash for unknown reasons (which is an extreme form of hallucinations) but I never experienced them to bs me. I guess I’ll have to reboot my computer every few hours to see if things improve.
The whole work reminds me of how I prepared in 1966-1969 for math olympiads in Poland. You can read about it here
Well, I was much younger then and now I do not have a river to go to and swim to get a break.
As soon as I finish this project I am going back to my own ideas of using AI in teaching of linear algebra as a continuation of