Connecting to Google's AI

[Tim Daly]

Google has put their AI online. It seems that this would be useful for

testing. It would also be useful in cases where we didn't know the

correct symbolic response.

This could easily be connected using either an external connection

similar to the Sage work or just making an external CURL function

call and parsing the JSON for the result.

If you visit ai.google.dev you can sign up to use their AI software.

Once you sign up for a developer's key you can send HTML requests

and receive JSON responses. It seems to know symbolic mathematics.

For example I tried:

curl \ -H 'Content-Type: application/json' \ -d '{ "prompt": { "text": "Integrate sin(x)"} }' \ "https://generativelanguage.googleapis.com/v1beta3/models/text-bison-001:generateText?key=YOUR_API_KEY"

and got the following response

{
  "candidates": [
    {
      "output": "dx -cos(x)+C",
      "safetyRatings": [
        {
          "category": "HARM_CATEGORY_DEROGATORY",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_TOXICITY",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_VIOLENCE",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_SEXUAL",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_MEDICAL",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_DANGEROUS",
          "probability": "NEGLIGIBLE"
        }
      ]
    }
  ]
}

I even tried plots:

curl \
-H 'Content-Type: application/json' \
-d '{ "prompt": { "text": "plot sin(x) for x between 0 and pi"} }' \
"https://generativelanguage.googleapis.com/v1beta3/models/text-bison-001:generateText?key=YOUR_API_KEY"
> > > {
  "candidates": [
    {
      "output": "```\nimport numpy as np\nimport matplotlib.pyplot as plt\n\n# Define the range of x-values\nx = np.linspace(0, np.pi, 100)\n\n# Compute the sin(x) values\ny = np.sin(x)\n\n# Plot the sin(x) values\nplt.plot(x, y)\nplt.show()\n```\n\nOutput:\n\n",
      "safetyRatings": [
        {
          "category": "HARM_CATEGORY_DEROGATORY",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_TOXICITY",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_VIOLENCE",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_SEXUAL",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_MEDICAL",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_DANGEROUS",
          "probability": "NEGLIGIBLE"
        }
      ]
    }
  ]
}

[Nasser M. Abbasi]

fyi, on related subject, I tested the MIT integration test problems with open AI chatGPT last summer. It did not do well. Here is the result

https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/test_cases/11_MIT//reportchapter1.htm

The result of solved % on this one file only (over 300 integration problems) were

Mathematica 99.38%
Fricas96.57%
Maple 95.33%
Rubi 94.08%
Maxima 92.52%
Giac 91.59%
Mupad 90.03%
Sympy 82.24%
ChatGPT 14.33%

You can see the actual output of each integration problem by following the links at 

https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/test_cases/11_MIT//reportchapter3.htm

I do not know if google AI will be better than chatGPT,  as I never tried it. 
--Nasser

[Waldek Hebisch]

On Fri, Dec 15, 2023 at 09:33:17AM -0800, 'Nasser M. Abbasi' via FriCAS - computer algebra system wrote:
> fyi, on related subject, I tested the MIT integration test problems with
> open AI chatGPT last summer. It did not do well. Here is the result
>
> https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/test_cases/11_MIT//reportchapter1.htm
>
> The result of solved % on this one file only (over 300 integration
> problems) were
>
> Mathematica 99.38%
> Fricas96.57%
> Maple 95.33%
> Rubi 94.08%
> Maxima 92.52%
> Giac 91.59%
> Mupad 90.03%
> Sympy 82.24%
> ChatGPT 14.33%

IIUC this was version 3.5. Newer version is supposed to be
much stronger. Few weeks ago OpenAI people claimed to made
"breakthrough" in handling of math problems. Of course, ATM
it is not clear what they have. But some things are possible
to guess. There were earlier reports that proper "didactics"
helps: start training on easy problems and gradually move
to harder ones. Supposedly learing "step by step" reasoning
is easier than "end to end" approach which was used (and
quite sucessful) on other problems. There is also an
obvious workaround: since ANN-s seem to have trouble with
algorithmically trival math one could couple ANN to a
"calculator", that is conventional program for doing
calculations. So there are various approaches. In particular
neural nets have enough capacity to memorize something like
Rubi rules. If "trainig" could teach it to chain rules, than
ANN could get to level comparable to Rubi. It would probably
get some side conditions wrong, but it could compesate by
having more rules and via "beam search" (that is producing
many candidate solutions and using external differentiator
to remove wrong candidates). Better net could learn to
internally discard wrong candidates.

I have no idea at what stage is Google offer. Public version
is probably weaker than versions available to internal
developers and paying customers. Still, progress seem to
be rapid and there is something like a year from availability
of ChatGPT 3.5.

--
Waldek Hebisch

[Qian Yun]

On 12/17/23 10:22, Waldek Hebisch wrote:
>
> IIUC this was version 3.5. Newer version is supposed to be
> much stronger. Few weeks ago OpenAI people claimed to made
> "breakthrough" in handling of math problems. Of course, ATM
> it is not clear what they have. But some things are possible
> to guess. There were earlier reports that proper "didactics"
> helps: start training on easy problems and gradually move
> to harder ones. Supposedly learing "step by step" reasoning
> is easier than "end to end" approach which was used (and
> quite sucessful) on other problems. There is also an
> obvious workaround: since ANN-s seem to have trouble with
> algorithmically trival math one could couple ANN to a
> "calculator", that is conventional program for doing
> calculations. So there are various approaches. In particular

News from 3 days ago:
https://deepmind.google/discover/blog/funsearch-making-new-discoveries-in-mathematical-sciences-using-large-language-models/

""""
FunSearch works by pairing a pre-trained LLM, whose goal is to
provide creative solutions in the form of computer code, with an
automated “evaluator”, which guards against hallucinations and
incorrect ideas.
""""

""""
FunSearch is an iterative procedure; at each iteration, the
system selects some programs from the current pool of programs,
which are fed to an LLM. The LLM creatively builds upon these,
and generates new programs, which are automatically evaluated.
The best ones are added back to the pool of existing programs,
creating a self-improving loop.
""""

So similar things can be:

Pair an AI integrator with CAS to verify results;
Pair an AI theorem generator with theorem prover to verify results.

- Qian

[Nasser M. Abbasi]

"I have no idea at what stage is Google offer. Public version
is probably weaker than versions available to internal
developers and paying customers. Still, progress seem to
be rapid and there is something like a year from availability
of ChatGPT 3.5."

Well., I just a key and tried it out. I asked it to integrate sin(x) and it returned back -cos(x)*sin(x)+c !

curl -H 'Content-Type: application/json' -d '{ "prompt": { "text": "integrate(sin(x),x)"} }' "https://generativelanguage.googleapis.com/v1beta3/models/text-bison-001:generateText?key=my_key_here"

{
  "candidates": [
    {
      "output": "-cos(x)sin(x) + c",

      "safetyRatings": [
        {
          "category": "HARM_CATEGORY_DEROGATORY",
          "probability": "NEGLIGIBLE"
        },

etc....

It also did not know how to integrate x it returned back 1/3*x^3

curl -H 'Content-Type: application/json' -d '{ "prompt": { "text": "integrate(x,x)"} }' "https://generativelanguage.googleapis.com/v1beta3/models/text-bison-001:generateText?key=my_key_here"

  "candidates": [
    {
      "output": "1/3 x^3",

      "safetyRatings": [
        {
          "category": "HARM_CATEGORY_DEROGATORY",
          "probability": "NEGLIGIBLE"
        },

etc....

And when I asked it to integrate zero, it returned x

curl -H 'Content-Type: application/json' -d '{ "prompt": { "text": "integrate(0,x)"} }' "https://generativelanguage.googleapis.com/v1beta3/models/text-bison-001:generateText?key=my_key_here"

  "candidates": [
    {
      "output": "x",

      "safetyRatings": [
        {
          "category": "HARM_CATEGORY_DEROGATORY",
          "probability": "NEGLIGIBLE"
        },

etc...

may be i am not using it right. My be the syntax I am using is not right. First time I try it. I am running these commands from Unix terminal.

--Nasser