By Milton Mueller on
Oct 11, 2026 10:03 am
On
October 8, the AI in Geopolitics Network held its inaugural
meeting at the Tufts
University Fletcher
School in Medford,
Massachusetts. The
network was formed by
researchers at MIT’s Future Tech project, the University of Illinois I-School, and the Fletcher School and includes IGP as a member.
The purpose of AIGEN is to bring
together policy
makers, scholars and
technologists in an
interdisciplinary
space to explore the
geopolitical
implications of a
globally distributed
digital computing
infrastructure and
artificial
intelligence
applications. I
profited immensely
from the mix of
policy
practitioners,
academics, business,
political economy
and cybersecurity
expertise.
Invited to speak on the opening panel
on “AI, AGI, and the
evolving landscape
of power,” my
comments, below,
focused on how
the public dialogue
about governance of
information technology
is distorted by the
myth of a
superintelligence that
can wipe out humanity.
AI, AGI, and the evolving landscape of
power
It
is hard to overstate
the damage done to AI
governance by the
story, the myth, of an
artificial
superintelligence that
can kill us all. In a
paper published a year
ago, I argued that AGI is an empty construct, and
that machine
intelligence, no
matter how “super,”
does not give it an
autonomous will, much
less the power to
destroy humanity. Others have published similar refutations.
Yet the myth persists.
Rejecting
the AGI myth doesn’t
mean, of course, that
humans can’t destroy
humanity. A nuclear
weapon escalation
cycle probably could
do that. The
difference is that we
all grasp that it’s
humans who are
responsible for
decisions about how
nukes are used and the
likely effects. The
AGI myth, however,
tells us to fear an
autonomous technology,
not our decisions
about how to use it.
The
tech’s evolution is
supposed to be
dangerous because of
Recursive
Self-Improvement, or
RSI. The doomers claim
that an information
system can RSI its way
to an autonomous will
and overwhelmingly
powerful capabilities.
But RSI only works if
the AI is given a
well-defined goal that
the machine learning
process can hone in
on. The AGI myth, in
contrast, says that
RSI can push the
machine into any and
all objectives and
indeed allow it to
define its own
objectives. This to my
mind is a pretty
elementary logical
error. It also ignores
the constraints of
physics.
Superintelligence is
the information
systems equivalent of
a perpetual motion
machine in
thermodynamics.
In
this talk, I want to
focus on a more
practical topic: how
the public dialogue
about governance of
information technology
is distorted by the
AGI threat, despite
its mythical nature.
A
rational policy
dialogue would see AI
as an extension of
computing and the
internet. It would see
continuity and
evolution, not
revolution. AI is
nothing but a souped
up software
application living off
a distributed,
ubiquitous and
powerful computational
infrastructure. Yes,
its computing
capabilities and
feedback loops can
produce unintended
effects and disrupt
established equilibria
in industry and other
social activities.
But, lest we forget,
so did the internet
and social media.
Almost all of the
policy problems
attributed to AI are
extensions of problems
attributed earlier to
internet connectivity,
web sites, search
engines, and social
media. All the fears
attributed to hacking
by “rogue agents” are
just a continuation of
cybersecurity
dialogues about
software
vulnerabilities,
viruses and worms that
go back to the release
of the Morris Worm in
1988. In a rational
policy dialogue, we
would analyze the
problems or harms
caused by specific ML
applications, and
focus on who’s going
to be liable for the
damage. Such a
dialogue would
encompass both
misaligned models and
their use by
misaligned humans.
Much of AI governance
would boil down to
adaptations of
existing ICT laws and
contracts regarding
product liability,
copyright
infringement, data
protection,
cybersecurity,
advertising, content
regulation, and so on.
If
the threat is
existential, on the
other hand, there is
no time for social
adaptation. The
apocalypse can only be
avoided by halting all
development, slowing
it down, or imposing
ex ante review of
frontier models. Here
again the specter of a
Superintelligence
lurks. Frontier means
“closer to
Superintelligence,” or
“on the edge of
explosive Recursive
Self-Improvement.”
Beyond the frontier
lies the Wild Indians
of uncontrolled
machine intelligence.
The further you go
past the frontier the
greater the danger.
Don’t we need a
gatekeeper?
Here
are 4 ways in which
AGI/SI sets policy on
the wrong track:
- It
diverts attention
from human agency,
human decisions,
human
responsibility. It’s
the machine itself
we have to worry
about.
- It
cuts us off from
learning and
precedent in digital
governance – from
seeing the
continuity and
lessons of Internet
and cybersecurity
governance.
- It
flattens a
heterogeneous set of
policy issues into a
single thing.
- It
creates an arms
race; it securitizes
or militarizes
digital governance.
On
#1, Let’s talk about
the Hugging Face
incident, and how AGI
Fears have distorted
our response to it.
Let’s compare Hugging
Face with the
CrowdStrike update
that shut down or
crashed approximately
8.5 million Microsoft
Windows computers
worldwide? Which one
was more costly and
damaging? CrowdStrike
by a mile! Was that an
AI governance incident
– a case of “rogue
agents”? No one seems
to think so, but it
was an automated
update program that
did things no one told
it to do. Delta
Airlines charges
CrowdStrike with gross
negligence,
CrowdStrike appeals to
contract-based
liability limits. May
the best lawyers win.
Whoever wins, here we
have a rational
exploration of human
(corporate)
responsibility, not
blathering about AI
agents taking over the
world.
On
#2, If we are trying
to make policy
regarding self-driving
cars, should we be
worried about whether
they are going to go
berserk and kill us
all, or should we be
focused on the
specific risks posed
by cars running into
each other or hitting
pedestrians? If we are
focused on existential
risk, how can we
effectively apply and
modify established
concepts of product
liability, traffic
management, and
transport
infrastructure?
On
#3, AI can be used for
medical diagnosis,
facial recognition,
sorting through job
applications, running
cars, guiding
missiles, spell
checking, image
generation, writing
student essays, and
thousands of other
things. Don’t these
applications all pose
very different public
policy issues? Does it
make any sense to talk
about “regulating” or
“not regulating” some
generality called
“AI?” And yet
superintelligence
fears collapse all
these things into one
bucket.
On
#4, the AGI myth tell
us that there is a
race on, that AGI is
the finish line, and
that the winner gets
hierarchical power
over everyone else. In
this view, winning an
AI race is just like
winning the race to
build an atomic bomb.
Indeed, the
singularity of the
Singularity, the idea
that RSI produces a
single
Superintelligence with
total control and no
competitors, is one of
the silliest yet least
challenged aspects of
the AGI story. The Big
Bang-like emergence of
AGI, however, is a
dead giveaway to the
essentially religious
mindset from which
these stories emerge.
AGI is not about
computer science, it’s
a cosmological origin
story for nerds.
The
problem with the “AI
race” narrative is
that it makes a
globalized digital
ecosystem an
instrument in a
nationalistic arms
race. The leading
American AI firms
(Anthropic, OpenAI,
Musk) advocate export
controls on China and
a nationally fenced
off AI industry. But
it’s not just the
model developers; it’s
chips and data and
everything else
digital on the line
here. If AI is a chess
piece in a
geopolitical power
race, the whole
Internet, all
applications running
on it, all digital
devices, all the data
they produce, must be
subject to national
security controls,
because all of these
things contribute to
AI development. We’ve
seen this taken to a
ridiculous degree by
the Trump and Biden
administrations. Look
at the FCC’s ban on
foreign home routers.
The intel community
supported the TikTok
ban because they
thought China could
use its user data to
feed AI applications
that could control our
minds. Dozens of other
restrictive actions
going back 10 years
and spanning
Presidential admins
from both parties,
have been based on the
premise that any
Chinese ICT equipment,
services, even
investment capital are
threats to our
security.
Purveyors
of the AI arms race
narrative face a big
problem, however. Most
of the world’s
production and
consumption of digital
information systems
are embedded in a
globalized market
economy. That
transnational economy
thrives precisely
because its activities
are less inhibited by
national borders and
the friction, taxes,
transaction costs, and
military risk that
goes with them. Those
non-jurisdictional
spaces evolved their
own governance
institutions, which
are mostly in the
hands of
nongovernmental forms
of collective action.
These governance
institutions are
somewhat insulated
from military conflict
and political coercion
by states, and are
more responsive to the
global public.
The
three basic policy
choices we need to
make are
- Whether
we see AI gov as a
continuation of
computing and ICT
policy
- Whether
ICT is governed
primarily by
civilian or military
logic
- Whether
we try to cram the
global Internet back
into a territorial
jurisdictional legal
framework, or
continue to reap the
economic growth,
improved access to
information and
wider distribution
of digital tools
associated with its
relative freedom.
The
post AI in Geopolitics Network appeared first on Internet Governance Project.
so we don't up in your
junk folder.
|
