Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Experiment with AI

6 views
Skip to first unread message

HRM Resident

unread,
Feb 27, 2023, 8:28:37 AM2/27/23
to

I did the following to further explore ChatGPT and test my theory
of its lack of accuracy and trustworthiness, as pointed out by Mike by the
poor response to the Latin version of Winnie The Pooh.

(1) I got a 1-month subscription to the latest version.

(2) I wrote a 45-line Python program to reformat some text and
tested it exhaustively, proving it worked as intended.

(3) I removed one line, causing it to provide incorrectly
formatted text.

(4) I submitted the faulty version to ChatGPT and described
the problem. I asked it to provide me with a working version.

It tried 8-9 times to give me a version that worked, failing
every time. Each version it provided was more complicated, made the
failure worse, and often was 2-3 times longer than the original. It
never got a working variant.

I believe it is not even close to being helpful, and it can't
'think' or display any characteristics of 'intelligence.'

James, I will discuss this with you if you are interested, but
if you want to 'mop the floor' with me or compare me to Carter
again, I will cease.

--
HRM Resident

lucr...@florence.it

unread,
Feb 27, 2023, 8:58:11 AM2/27/23
to
On Mon, 27 Feb 2023 09:28:35 -0400, HRM Resident <hrm...@gmail.com>
wrote:

>
> It tried 8-9 times to give me a version that worked, failing

Does this give new meaning to the term Arguebot???? :-®

HRM Resident

unread,
Feb 27, 2023, 9:32:03 AM2/27/23
to
lucr...@florence.it writes:

>
> Does this give new meaning to the term Arguebot???? :-®

Yes, in a slighly different way. It is a computer program that
incessantly replies, but seems to dig itself deeper into a hole each
time.

Because our 'Arguebot' is human and can intelligently think, he is
usually correct, or at worst, keeps an argument/debate going for
entertainment.

ChatGPT makes maistakes, and when they are explained in
detail, it fails to give a better explanation. It just enhances the
wrong answer. I've spent a couple of weeks testing it, and correcting
it, and it doesn't 'learn from its mistakes.'

--
HRM Resident

James Warren

unread,
Feb 27, 2023, 11:10:35 AM2/27/23
to
I have read reports of people using chatGPT to write programs.
It cannot be trusted but if used with some guidance and many
iterations it can eventually produce working code. In a few cases
even better code than the programmer.

I don't think it will work woth the instruction "this code is broken, fix it".
You need to tell it exactly what you want the code to do.

In my case it produced incomplete code which would work if
I provided the correct functions which it on provided a stub for.
I suspect if I had been more persistent it might have eventually
produced a decent program. Then again it might have been limited
to the resources it provided for free.

In other areas I got it to produce contradictions in logic which
I suggested was because it had policies to prevent offending
people's political or religious beliefs. If these policies were
removed it would be more useful. Perhaps a paid version
would not have these policies. Try it. See if it will contradict
itself.

Of course, many people have found it makes mistakes and it
even lies. This might be because it tries to be too agreeable.
It passes medical exams, law exams and business exams
but only at the C or B level. Whatever it says should be independently
verified. C or B level expertise is not good enough to be reliable.

HRM Resident

unread,
Feb 27, 2023, 12:13:14 PM2/27/23
to
James Warren <jwwar...@gmail.com> writes:

>
> I have read reports of people using chatGPT to write programs.
> It cannot be trusted but if used with some guidance and many
> iterations it can eventually produce working code. In a few cases
> even better code than the programmer.
>
> I don't think it will work woth the instruction "this code is broken, fix it".
> You need to tell it exactly what you want the code to do.

I am using the paid version (one month only.) It's still
overworked in that many times I still can't get on, or it stalls in
mid-session. However, the paid version seems up-to-date and not
limited to what it knew in 2021.

It's far from ready to be turned loose on the general public.
The problem I gave it was simple. I wrote a Python program that
reformatted text. I deleted one line of code that indented the first
line of paragraphs by 4 characters. My question was, "Can you
refactor this code to indent the first line of each paragraph with 4
spaces?" It couldn't. It gave me back versions that made every
paragraph one line. It gave me back versions that indented every
line but the first one. Each time, I carefully said, "No, that's not
correct. It needs to indent the first line of each paragraph by 4
characters, not <whatever it did>."

As I previously said, most versions it provided increased in
length and complexity. Python is the most widely used language
today, and it ought to be better with it than other languages. It
can spit out known procedures like FFTs, sorting algorithms,
statistics packages, etc., that are everywhere on the Internet and
in public code libraries. However, it can't handle new stuff. As a
result, our code or shell scripts with simple bugs stump it most of
the time. If I need to be better at whatever I ask it to check, I
wonder if it's helpful.

I agree it's "too polite" and never gives up. It can't seem to
say, "That's something I'm still working on, and I can't answer your
question yet." We agreed months ago, "No result is better than a bad
result." It doesn't have that capacity.

I didn't try to trick it with logic questions. Instead, I wanted
to use it to debug and optimize code. I tried this for 2 weeks with
various programs and scripts. 90% or more of the time, it failed.

--
HRM Resident

James Warren

unread,
Feb 27, 2023, 12:35:23 PM2/27/23
to
That's not necessarily true. A bad result can be corrected or refined.

>
> I didn't try to trick it with logic questions. Instead, I wanted
> to use it to debug and optimize code. I tried this for 2 weeks with
> various programs and scripts. 90% or more of the time, it failed.
>
> --
> HRM Resident

I asked it for programs to do logistic regression and back propagation.
What it gave was correct but incomplete. For the backprop it wanted
more detailed specifications. Since I didn't have a specific structure
in mind it could not complete my request.

I saw a video of a guy trying to get it to design a webpage in Java. It took
many iterations and refinements to get it right. At best it is an assistant
not a substitute for a programmer. As such, it might be able to speed up
the development process.

YMMV always applies, I guess.

HRM Resident

unread,
Feb 27, 2023, 1:50:44 PM2/27/23
to
James Warren <jwwar...@gmail.com> writes:

>
> That's not necessarily true. A bad result can be corrected or refined.
>
You and I can. ChatGPT seems unable to be corrected by users. I
asked it an amateur radio-related question that I knew the answer
to. Unfortunately, it got it completely wrong. I told it the
response needed to be corrected and gave it the right one. It
thanked me and said it would use the corrected version. I then
logged out and back in and asked the same question. It gave me the
updated information.

The next day I asked the question again, and it responded with
the initial, incorrect response. I was right because I was involved
in the topic, and it's easily verified on the Internet. Yet it seems
determined not to be corrected, at least in that case.

>
> I saw a video of a guy trying to get it to design a webpage in Java. It took
> many iterations and refinements to get it right. At best it is an assistant
> not a substitute for a programmer. As such, it might be able to speed up
> the development process.
>
That was my initial thinking. Then, instead of a team of
programmers in a software shop, a company could hire one or two top-
drawer coders to ask the right questions. ChatGPT or equivalent
would provide the programs or subroutines, eliminating the need for
most of the "code monkeys." Based on my two weeks of asking many
varied software questions, the coders have job security for the
foreseeable future.

--
HRM Resident

James Warren

unread,
Feb 27, 2023, 2:37:43 PM2/27/23
to
There are already several coding assistants. chatGPT is as good as
many of them.

https://sourceforge.net/software/ai-coding-assistants/
0 new messages