[ PRIVACY Forum ] The dangers of AI coding

[Lauren Weinstein]

The dangers of AI coding

I am SO glad I phased out of most coding years ago, except as needed
for my own systems. Those jobs are toast. But the dangers are very
real.

Just now I needed a Bash script for a network monitoring task. I must
have written dozens of these in various forms over the years. Pings
and status flags and the usual stuff.

So this time, just for the hell of it, I asked Gemini (free version of
course) to do it:

"write me a bash script that will ping a specific ip address and when
the pings start failing keep trying to ping and then when the pings
are successful again send a specific curl command to that ip address"

And about 10 seconds or less later out came a completely reasonable
looking, nicely commented Bash script, along with a reminder to make
the file executable and how to stop it with ^C.

This of course is a very simple, really trivial task, and I was able
to quickly read through the code and verify that it looked correct.

The problem of course is obvious. I could only do this verification
because I have enough skill to easily write that code MYSELF, it would
just take me more time. If the code were more complex and/or
voluminous, just checking could range from very lengthy to utterly
impractical to do at all, meaning any errors could go undetected with
everything that implies, especially for dangerous "sleeper" bugs.

There may be a useful analogy to vehicle driver assist systems, that
may lull drivers into being less attentive and causing them to be
unable to respond to emergency situations quickly when their
intervention is most required.

Crashing code and crashing cars. All very dangerous.

L

- - -
--Lauren--
Lauren Weinstein
lau...@vortex.com (https://www.vortex.com/lauren)
Lauren's Blog: https://lauren.vortex.com
Mastodon: https://mastodon.laurenweinstein.org/@lauren
Signal: By request on need to know basis
Founder: Network Neutrality Squad: https://www.nnsquad.org
PRIVACY Forum: https://www.vortex.com/privacy-info
Co-Founder: People For Internet Responsibility
_______________________________________________
privacy mailing list
https://lists.vortex.com/mailman/listinfo/privacy