Slouching towards the singularity
Aaron Bergman
computers stopped getting faster a few years ago. Instead, they're
getting more parallel, and that's far from the same thing. They're also
getting hotter. A lot hotter. You want to defeat skynet? Knock out the
cooling system, and the rest will take care of itself.
Aaron
Mike Ash
Aaron Bergman <aber...@physics.utexas.edu> wrote:
More parallel is not different from faster, it's merely a different FORM
of faster than what we saw before. Computers are still getting massively
faster all the time, it's just that software which isn't written to take
advantage of the new kind of faster remains slow.
This change shouldn't affect the drive for AI (or intelligence
augmentation), which is mostly what the Singularity is about. Notice
that the human brain is a massively parallel system of very slow
processors. This proves that there's nothing inherently serial about
human-level thought.
Hotter is not really right either. What's changed is our tolerance for
heat. In terms of performance per watt, (or equivalently, computation
per joule) today's computers are still massively better than anything in
the past.
In short, while I'm sure that singularity writers have noticed the
changes in the computer industry in the last few years, it doesn't
really change what they're writing about.
--
Mike Ash
Radio Free Earth
Broadcasting from our climate-controlled studios deep inside the Moon
A.G.McDowell
advantage over people that the best way to solve X, for any X, is to get
a computer to do it for you. We presume that the human brain derives its
intelligence from massive parallelism because the underlying components
are pitifully slow, in computer terms, so a brick wall in computer clock
speed is not an overwhelming obstacle to this.
OTOH another component is the idea that better computers building better
computers building better computers will suddenly solve every problem in
sight. It is not obvious to me that massive increases in intelligence
will necessarily solve world-shaking problems. Intractable problems
might have an optimum solution not much better than the current sub-
optimal approach, might require data without which no amount of
intelligence can do any more than outline a large set of possible right
answers, or might be bound by limits in theoretical computing
(especially if P != NP, as many experts believe). Also, many
intellectual tools whose discovery required genius - statistics and
experimental design, axiomatic approaches, appeals to symmetry - are now
routinely taught to mortals below genius grade, so (in relative terms)
de-skilling scientific discovery. Silicon or carbon based super-geniuses
will certainly extend the region lit by light of our intelligence, but
until that happens, I don't think we will know whether what is revealed
is a garden or a desert.
--
A.G.McDowell
Eivind
> I wonder if any people writing about the singularity have noticed that
> computers stopped getting faster a few years ago. Instead, they're
> getting more parallel, and that's far from the same thing.
It's a very similar thing. There's a few problems where being more
parallell doesn't help you much, but those are the exception, certainly
simulating a million or a billion neurons is a lot quicker with 100
cores than with 1 core.
> They're also getting hotter. A lot hotter.
Bull. They're producing less and less heat for the same amount of
computation. The hottest processors is just a function of
engineering-constraints and how much poweruse matters in the
application. (sometimes using twice the power to gain 30% performance is
worth it, sometimes it's not)
Eivind
Aaron Bergman
Mike Ash <mi...@mikeash.com> wrote:
> In article <abergman-22A3BE...@news.tamu.edu>,
> Aaron Bergman <aber...@physics.utexas.edu> wrote:
>
> > I wonder if any people writing about the singularity have noticed that
> > computers stopped getting faster a few years ago. Instead, they're
> > getting more parallel, and that's far from the same thing. They're also
> > getting hotter. A lot hotter. You want to defeat skynet? Knock out the
> > cooling system, and the rest will take care of itself.
>
> More parallel is not different from faster, it's merely a different FORM
> of faster than what we saw before. Computers are still getting massively
> faster all the time, it's just that software which isn't written to take
> advantage of the new kind of faster remains slow.
It's not just a matter of taking advantage of increased parallelism,
it's dealing with the memory issues that gets you.
>
> This change shouldn't affect the drive for AI (or intelligence
> augmentation), which is mostly what the Singularity is about. Notice
> that the human brain is a massively parallel system of very slow
> processors. This proves that there's nothing inherently serial about
> human-level thought.
>
> Hotter is not really right either. What's changed is our tolerance for
> heat. In terms of performance per watt, (or equivalently, computation
> per joule) today's computers are still massively better than anything in
> the past.
Even that has slowed down or stopped as I understand it. All that's
really going on is that feature size is decreasing allowing more and
more cores to be placed on a chip. If you work out the power and cooling
requirements for a really large computer (including heat dissipated from
memory, etc), it can get rather high. The current top computers are
already drawing on the order of 10 MW.
>
> In short, while I'm sure that singularity writers have noticed the
> changes in the computer industry in the last few years, it doesn't
> really change what they're writing about.
The point I wanted to make is that it's not a straight line from here to
faster computers. The ramp up in clock speed is done, and while the ramp
up in feature size continues, that doesn't translate directly into
speed. Some fundamental advances will have to be made so that computers
will continue to speed up. It's also worth remembering that lots of
things thought to be exponential are really just the first part of
logistic curves.
Aaron
Howard Brazee
>More parallel is not different from faster, it's merely a different FORM
>of faster than what we saw before. Computers are still getting massively
>faster all the time, it's just that software which isn't written to take
>advantage of the new kind of faster remains slow.
Brains aren't faster than computers - but they are very, very
parallel.
--
"In no part of the constitution is more wisdom to be found,
than in the clause which confides the question of war or peace
to the legislature, and not to the executive department."
- James Madison
Mike Ash
Howard Brazee <how...@brazee.net> wrote:
> On Thu, 22 Oct 2009 00:31:57 -0400, Mike Ash <mi...@mikeash.com> wrote:
>
> >More parallel is not different from faster, it's merely a different FORM
> >of faster than what we saw before. Computers are still getting massively
> >faster all the time, it's just that software which isn't written to take
> >advantage of the new kind of faster remains slow.
>
> Brains aren't faster than computers - but they are very, very
> parallel.
Brains ARE much faster than computers. Glance at a scene, tell me what's
in it. You can do this way faster than any computer. That we have bad
software for basic operations like multiplication does not change this
fact.
Again, parallel is not different from faster, it's just a different KIND
of faster.
Mike Ash
Aaron Bergman <aber...@physics.utexas.edu> wrote:
> In article <mike-EF00C2.0...@news.eternal-september.org>,
> Mike Ash <mi...@mikeash.com> wrote:
>
> > In article <abergman-22A3BE...@news.tamu.edu>,
> > Aaron Bergman <aber...@physics.utexas.edu> wrote:
> >
> > > I wonder if any people writing about the singularity have noticed that
> > > computers stopped getting faster a few years ago. Instead, they're
> > > getting more parallel, and that's far from the same thing. They're also
> > > getting hotter. A lot hotter. You want to defeat skynet? Knock out the
> > > cooling system, and the rest will take care of itself.
> >
> > More parallel is not different from faster, it's merely a different FORM
> > of faster than what we saw before. Computers are still getting massively
> > faster all the time, it's just that software which isn't written to take
> > advantage of the new kind of faster remains slow.
>
> It's not just a matter of taking advantage of increased parallelism,
> it's dealing with the memory issues that gets you.
Can you elaborate on this? As far as I understand, the only "memory
issues" are that it's gets hard to provide a uniform memory space for
all your processors as you get more of them, which means that you tend
to have local memory for each one (or for groups of them), which in turn
means that you need to write code that's aware of this locality. But
this is just a facet of "taking advantage of increased parallelism".
> > This change shouldn't affect the drive for AI (or intelligence
> > augmentation), which is mostly what the Singularity is about. Notice
> > that the human brain is a massively parallel system of very slow
> > processors. This proves that there's nothing inherently serial about
> > human-level thought.
> >
> > Hotter is not really right either. What's changed is our tolerance for
> > heat. In terms of performance per watt, (or equivalently, computation
> > per joule) today's computers are still massively better than anything in
> > the past.
>
> Even that has slowed down or stopped as I understand it. All that's
> really going on is that feature size is decreasing allowing more and
> more cores to be placed on a chip. If you work out the power and cooling
> requirements for a really large computer (including heat dissipated from
> memory, etc), it can get rather high. The current top computers are
> already drawing on the order of 10 MW.
This is not true at all. You need to remember that there's more to the
world of computation than CPUs made by Intel. Graphics processors are
where the huge advances in parallel computation are being made. For
example, an ATi Radeon HD 2350 from 2007 computes at 42 GFLOPS and
consumes 35W of power. A Radeon HD 5870, released last month, does 2720
GFLOPS. Following a strict linear performance/watt relation, you'd
expect the 5870 to consume over 3 kilowatts of power. The actual figures
are 188W at full blast, and 27W when idle.
> > In short, while I'm sure that singularity writers have noticed the
> > changes in the computer industry in the last few years, it doesn't
> > really change what they're writing about.
>
> The point I wanted to make is that it's not a straight line from here to
> faster computers. The ramp up in clock speed is done, and while the ramp
> up in feature size continues, that doesn't translate directly into
> speed. Some fundamental advances will have to be made so that computers
> will continue to speed up. It's also worth remembering that lots of
> things thought to be exponential are really just the first part of
> logistic curves.
It never has been a straight line. Moore's Law has been five years away
from ending for as long as I can remember. It was never an automatic
process that was guaranteed to keep happening year over year. Every
doubling has involved massive amounts of research and technological
breakthroughs. This was as true in 1989 as it is in 2009.
trag
I for one welcome our new paralllel masters...
William December Starr
trag <tr...@io.com> said:
> I for one welcome our new paralllel masters...
May they never meet, except at infinity.
-- wds
PV
>I wonder if any people writing about the singularity have noticed that
>computers stopped getting faster a few years ago.
Wrong. They aren't doubling in speed, but they're still speeding up
individually.
>Instead, they're getting more parallel, and that's far from the same thing.
Moore's law disagrees with you. There may be some danger of the real
measure of moore's law (transistor density) being beaten in a few years,
but people have been saying that for several decades and we keep finding
new tricks.
>They're also getting hotter. A lot hotter.
Not really - your information is a bit out of date.
>You want to defeat skynet? Knock out the
>cooling system, and the rest will take care of itself.
Skynet is distributed. Knock out *which* cooling system? *
--
* PV Something like badgers, something like lizards, and something
like corkscrews.
PV
>It's not just a matter of taking advantage of increased parallelism,
>it's dealing with the memory issues that gets you.
"Gets you". Really.
>Even that has slowed down or stopped as I understand it. All that's
You understand wrongly. *
PV
>sight. It is not obvious to me that massive increases in intelligence
>will necessarily solve world-shaking problems. Intractable problems
Imagine if you could do many detailed simulations in real time of the
implications of every possible decision. That's what massively parallel
intelligence gets you. *
Joseph Nebus
>"A.G.McDowell" <mcdo...@nospam.co.uk> writes:
>>sight. It is not obvious to me that massive increases in intelligence
>>will necessarily solve world-shaking problems. Intractable problems
>Imagine if you could do many detailed simulations in real time of the
>implications of every possible decision. That's what massively parallel
>intelligence gets you. *
And then mercifully when the Singularity arrives we can leave the
massive simulations of whatever we were thinking about doing to do that
instead, and we can stay home catching up on what the Tivo's recorded and
maybe finishing the Strategic Book Reserve pile that's now taller than
the luggage.
--
Joseph Nebus
------------------------------------------------------------------------------
Scott Lurndal
>Aaron Bergman <aber...@physics.utexas.edu> writes:
>>It's not just a matter of taking advantage of increased parallelism,
>>it's dealing with the memory issues that gets you.
>
>"Gets you". Really.
Not sure what point you're attempting to make, but certainly as a system
scales, NUMA (Non-Uniform Memory Access) issues begin to dominate. Fortunately
most modern operating systems (Windows, Linux) have internal support
such that they attempt to make intelligent scheduling and memory allocation
decisions on behalf of non-numa-aware applications.
Adding application awareness of NUMA can make a huge difference in
performance on large-scale ccNUMA systems (a classic example is the
Gaussian application. G09 is the first NUMA-aware release and
performance now scales pretty much linearly as nodes are added. Prior
versions of Gaussian (G03) did not scale well at all on large NUMA system).
Current server processor packages, btw, have been stuck at about
100 watts TDP for the last several generations. The vendors bin
various parts at various speeds that offer both lower and higher
TDP, but the most common parts sit at 90-120 watts per socket.
In a traditional non-virtualized data center, the servers run at
an average of less than 20% utilization year-over-year, so they
end up dissipating a lot of heat but producing relatively little
light[*]. Virtualization is changing that and starting to drive
utilization closer to 60-70%. Don't want to go much above that,
as there is a knee in the throughput curve at about 75% utilization.[**]
scott
[*] ACPI does provide some mechanisms to reduce the heat load at
idle, but consolidation/virtualization results in much better power, space
and cooling utilization.
[**] and of course virtualization effectiveness is predicated on the
virtual workloads not all peaking at the same time. Historically,
workloads "followed the sun", but globalization is changing that too.
trag
> In article <bcdcf794-fbb9-4926-a0b6-2a6763a97...@l35g2000vba.googlegroups.com>,
> trag <t...@io.com> said:
>
> > I for one welcome our new paralllel masters...
>
> May they never meet, except at infinity.
Hmmm. I wonder how the Hounds of Tindalos (sp?) feel about that.
Erik Max Francis
Ah, but we do not live in a Euclidean universe, so that postulate is out
the window ...
--
Erik Max Francis && m...@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 18 N 121 57 W && AIM/Y!M/Skype erikmaxfrancis
Diplomacy and defense are not substitutes for one another. Either
alone would fail. -- John F. Kennedy, 1917-1963
Erik Max Francis
> Aaron Bergman <aber...@physics.utexas.edu> writes:
>> I wonder if any people writing about the singularity have noticed that
>> computers stopped getting faster a few years ago.
>
> Wrong. They aren't doubling in speed, but they're still speeding up
> individually.
>
>> Instead, they're getting more parallel, and that's far from the same thing.
>
> Moore's law disagrees with you.
How does a noticed empirical relationship "disagree" with anybody? In
Moore's original paper, he simply noted the relationship, and made a
casual prediction that it would continue for at least another decade --
until 1975:
ftp://download.intel.com/museum/Moores_Law/Articles-Press_Releases/Gordon_Moore_1965_Article.pdf
Aaron's observation is right. Raw chip CPU speed has not increased
significantly over the last five years or so. What has increased is the
number of CPU cores per chip, and also heat dissipation problems, as
anyone who uses computers in areas without air conditioning can attest to.
William December Starr
pv+u...@pobox.com (PV) said:
> Aaron Bergman <aber...@physics.utexas.edu> writes:
>
>> You want to defeat skynet? Knock out the
>> cooling system, and the rest will take care of itself.
>
> Skynet is distributed.
Only according to a movie that doesn't exist.
> Knock out *which* cooling system? *
-- wds
Howard Brazee
>> Brains aren't faster than computers - but they are very, very
>> parallel.
>
>Brains ARE much faster than computers. Glance at a scene, tell me what's
>in it. You can do this way faster than any computer. That we have bad
>software for basic operations like multiplication does not change this
>fact.
>
>Again, parallel is not different from faster, it's just a different KIND
>of faster.
I could have argued about this with the original poster, but decided
to go with his premise for a moment to show that his conclusion did
not follow.
Aaron Bergman
Mike Ash <mi...@mikeash.com> wrote:
> In article <abergman-7DB9E4...@news.tamu.edu>,
> Aaron Bergman <aber...@physics.utexas.edu> wrote:
>
> > In article <mike-EF00C2.0...@news.eternal-september.org>,
> > Mike Ash <mi...@mikeash.com> wrote:
> >
> > > In article <abergman-22A3BE...@news.tamu.edu>,
> > > Aaron Bergman <aber...@physics.utexas.edu> wrote:
> > >
> > > > I wonder if any people writing about the singularity have noticed that
> > > > computers stopped getting faster a few years ago. Instead, they're
> > > > getting more parallel, and that's far from the same thing. They're also
> > > > getting hotter. A lot hotter. You want to defeat skynet? Knock out the
> > > > cooling system, and the rest will take care of itself.
> > >
> > > More parallel is not different from faster, it's merely a different FORM
> > > of faster than what we saw before. Computers are still getting massively
> > > faster all the time, it's just that software which isn't written to take
> > > advantage of the new kind of faster remains slow.
> >
> > It's not just a matter of taking advantage of increased parallelism,
> > it's dealing with the memory issues that gets you.
>
> Can you elaborate on this? As far as I understand, the only "memory
> issues" are that it's gets hard to provide a uniform memory space for
> all your processors as you get more of them, which means that you tend
> to have local memory for each one (or for groups of them), which in turn
> means that you need to write code that's aware of this locality. But
> this is just a facet of "taking advantage of increased parallelism".
Writing locality aware code is hard. As I recall, there are algorithms
(and remember, I'm talking about things that will make use of all the
processing power, not laptop stuff) that just don't have the needed
locality. Of course, what this might mean -- and I probably should have
said this in the original post, but wanted to be a bit provocative -- is
that programs on massively parallel machines may end up acting a lot
like the human brain just out of sheer necessity.
>
> > > This change shouldn't affect the drive for AI (or intelligence
> > > augmentation), which is mostly what the Singularity is about. Notice
> > > that the human brain is a massively parallel system of very slow
> > > processors. This proves that there's nothing inherently serial about
> > > human-level thought.
> > >
> > > Hotter is not really right either. What's changed is our tolerance for
> > > heat. In terms of performance per watt, (or equivalently, computation
> > > per joule) today's computers are still massively better than anything in
> > > the past.
> >
> > Even that has slowed down or stopped as I understand it. All that's
> > really going on is that feature size is decreasing allowing more and
> > more cores to be placed on a chip. If you work out the power and cooling
> > requirements for a really large computer (including heat dissipated from
> > memory, etc), it can get rather high. The current top computers are
> > already drawing on the order of 10 MW.
>
> This is not true at all. You need to remember that there's more to the
> world of computation than CPUs made by Intel. Graphics processors are
> where the huge advances in parallel computation are being made. For
> example, an ATi Radeon HD 2350 from 2007 computes at 42 GFLOPS and
> consumes 35W of power. A Radeon HD 5870, released last month, does 2720
> GFLOPS. Following a strict linear performance/watt relation, you'd
> expect the 5870 to consume over 3 kilowatts of power. The actual figures
> are 188W at full blast, and 27W when idle.
GPUs are cool, but I was talking about CPUs. In general, I think there
are some fundamental limits about how much you can lower the voltage
thresholds, and I think we're getting pretty close.
>
> > > In short, while I'm sure that singularity writers have noticed the
> > > changes in the computer industry in the last few years, it doesn't
> > > really change what they're writing about.
> >
> > The point I wanted to make is that it's not a straight line from here to
> > faster computers. The ramp up in clock speed is done, and while the ramp
> > up in feature size continues, that doesn't translate directly into
> > speed. Some fundamental advances will have to be made so that computers
> > will continue to speed up. It's also worth remembering that lots of
> > things thought to be exponential are really just the first part of
> > logistic curves.
>
> It never has been a straight line. Moore's Law has been five years away
> from ending for as long as I can remember.
Who's talking about Moore's Law? Feature sizes are going to keep getting
smaller. The point I'm making is that more parallel doesn't necessarily
mean faster.
Aaron
Mike Ash
Aaron Bergman <aber...@physics.utexas.edu> wrote:
Yes, and writing parallel code in general is hard. Like I said,
non-locality of memory is pretty much an inherent part of parallelism,
not an extra thing in addition to it.
I know you were talking about CPUs and not GPUs, I just don't know why.
A modern GPU is fully general-purpose. Anything you can compute on a
CPU, you can compute on a GPU. Of course GPUs have different tradeoffs,
so it might end up being much slower, but this is no surprise. GPUs are
where the major advancements are coming right now, so GPUs are where you
should be looking.
> > > > In short, while I'm sure that singularity writers have noticed the
> > > > changes in the computer industry in the last few years, it doesn't
> > > > really change what they're writing about.
> > >
> > > The point I wanted to make is that it's not a straight line from here to
> > > faster computers. The ramp up in clock speed is done, and while the ramp
> > > up in feature size continues, that doesn't translate directly into
> > > speed. Some fundamental advances will have to be made so that computers
> > > will continue to speed up. It's also worth remembering that lots of
> > > things thought to be exponential are really just the first part of
> > > logistic curves.
> >
> > It never has been a straight line. Moore's Law has been five years away
> > from ending for as long as I can remember.
>
> Who's talking about Moore's Law? Feature sizes are going to keep getting
> smaller. The point I'm making is that more parallel doesn't necessarily
> mean faster.
Uh, Moore's Law is that whole ramp of computers getting ever faster that
you've been talking about this whole time. And I know what point you're
making, I just disagree that it makes any kind of sense. More parallel
*is* faster. If Computer A has twice as many processing cores as
Computer B and each core has the same processing power, then Computer A
is faster, pretty much by definition. That it will run some software no
faster than Computer B would is true but irrelevant. This is especially
true when you're talking about the Singularity, where the major event is
the creation of superhuman AI, which is virtually certain to be a
problem which parallelizes well.
Wayne Throop
: Who's talking about Moore's Law? Feature sizes are going to keep getting
: mean faster.
Yeah, it just means more things can be computed per given time interval.
Which scarcely resembles "faster" at all.
Wayne Throop thr...@sheol.org http://sheol.org/throopw
ZnU
"A.G.McDowell" <mcdo...@nospam.co.uk> wrote:
Of course ubiquitous sensor networks and sentient data mining tools
help with this aspect of the problem.
> or might be bound by limits in theoretical computing (especially if P
> != NP, as many experts believe). Also, many intellectual tools whose
> discovery required genius - statistics and experimental design,
> axiomatic approaches, appeals to symmetry - are now routinely taught
> to mortals below genius grade, so (in relative terms) de-skilling
> scientific discovery. Silicon or carbon based super-geniuses will
> certainly extend the region lit by light of our intelligence, but
> until that happens, I don't think we will know whether what is
> revealed is a garden or a desert.
It's worth reflecting that there are a large number of problems on
which even merely human level intelligences could probably make vastly
more progress than human civilization currently does... because
civilization presently allocates relatively few resources to these
projects due to the high cost of humans. A PhD-level researcher, with
overhead, benefits, costs... what? $150K/year? Probably more.
What if a machine with that intellectual capability were a $1000
off-the-shelf item, and you could rack them up in data centers? Oh, and
they'd have the equivalent of high-bandwidth telepathy for exchanging
information with each other, they'd have low-latency access to
the entire sum total of human knowledge, they'd be linked to
high-performance simulation engines that would often remove the delays
involved with running actual physical experiments, and teaching them to
understand new subjects would take seconds or minutes, rather
than years. Plus, they'd work 24/7 and never get distracted. And maybe
next year's model, while it not actually being any smarter, thinks twice
as fast....
Global R&D output could easily end up thousands of times higher than it
presently is.
--
"The game of professional investment is intolerably boring and over-exacting to
anyone who is entirely exempt from the gambling instinct; whilst he who has it
must pay to this propensity the appropriate toll." -- John Maynard Keynes
David Johnston
I kind of suspect they'd be more concerned with their problems than
ours.
ZnU
Aaron Bergman <aber...@physics.utexas.edu> wrote:
Consult this handy chart:
http://en.wikipedia.org/wiki/File:PPTMooresLawai.jpg
The silicon microprocessor is the fifth physical computing substrate
widely employed by humanity (not counting meat, anyway), and Moore's Law
appears to have been with us essentially since the beginning. There's no
particular reason to think it will end when silicon microprocessor
technology has run its course.
(Actually, there *is* a bit of a logistic curve on that chart. It
appears to be bending... up.)
ZnU
David Johnston <da...@block.net> wrote:
I was assuming that they'd be engineered not to be; that they'd be
intelligent, but without much in the way of personality or outside
interests.
But even if they weren't designed this way (maybe you just can't design
human-level intelligences like that, or maybe we'd decide it would be
immoral), and they were granted legal equality (so you couldn't just use
them as slave labor), they'd probably still be a lot cheaper to hire
than human PhDs presently are, as a simple matter of supply and demand.
At well under 1% of the production cost of a human PhD, civilization
could make a lot more of them. And they wouldn't need to pay off student
loans. Or, you know, eat.
Aaron Bergman
wrote:
> : Aaron Bergman <aber...@physics.utexas.edu>
> : Who's talking about Moore's Law? Feature sizes are going to keep getting
> : smaller. The point I'm making is that more parallel doesn't necessarily
> : mean faster.
>
> Yeah, it just means more things can be computed per given time interval.
> Which scarcely resembles "faster" at all.
If all you want to do is ray tracing, that's fine. If you want to
something more complicated (that requires random access to a large
amount of memory, say), you're going to have to work hard to make sure
all those cores aren't sitting idle waiting for data.
Aaron
David DeLaney
>Aaron Bergman <aber...@physics.utexas.edu>
>: Who's talking about Moore's Law? Feature sizes are going to keep getting
>: smaller. The point I'm making is that more parallel doesn't necessarily
>: mean faster.
>
>Yeah, it just means more things can be computed per given time interval.
>Which scarcely resembles "faster" at all.
" Early in the year 2080, a milestone is passed.
More events are occurring in the Wind than in Realtime:
The Inside has become larger than the Outside. "
Dave
--
\/David DeLaney posting from d...@vic.com "It's not the pot that grows the flower
It's not the clock that slows the hour The definition's plain for anyone to see
Love is all it takes to make a family" - R&P. VISUALIZE HAPPYNET VRbeable<BLINK>
http://www.vic.com/~dbd/ - net.legends FAQ & Magic / I WUV you in all CAPS! --K.
Robert Martinu
The application in question was an A.I. - those are by definition a
parallel thing. Its about processing input from various sources,
crosslinking that massive amounts of prior experiences, exploring "what
if"-scenarios. Even following imperative design patterns that all
parallel. Now put that into a neuronal network...
Most of the cores wouldn't even have to work with the exact data sets.
Give them a somewhat more general dataset and reeavluate only solutions
that sound promising with the actual input. You even get responses that
have a high chance of being reusable for future similar situation.
Should you encounter other inputs with a low hammond-distance to already
solved cases you know where to start. Your machine actually learns
better from experience explored because it can't supply all cores with
complete data.
Sure, we're getting far away from the predictability of current
software, but thats to be expected with A.I..
Robert Martinu
>> Skynet is distributed.
>
> Only according to a movie that doesn't exist.
Without distribution a single blow could knock it out.
Who would be stupid enough to tie all his resources to a single point of
failure? So, if they where good enough to build skynet, they probably
made the introductionary courses about fault tolerant systems.
Mike Ash
ZnU <z...@fake.invalid> wrote:
> Consult this handy chart:
> http://en.wikipedia.org/wiki/File:PPTMooresLawai.jpg
>
> The silicon microprocessor is the fifth physical computing substrate
> widely employed by humanity (not counting meat, anyway), and Moore's Law
> appears to have been with us essentially since the beginning. There's no
> particular reason to think it will end when silicon microprocessor
> technology has run its course.
>
> (Actually, there *is* a bit of a logistic curve on that chart. It
> appears to be bending... up.)
Since the question at hand is what's been happening with computer speeds
in the last 10 years or so, this chart, which ends in 2000, is pretty
much pointless. I have no doubt that it would continue the trend if you
added 2000-2009 to it, but as it stands that chart doesn't quite do the
job.
Howard Brazee
>Uh, Moore's Law is that whole ramp of computers getting ever faster that
>you've been talking about this whole time. And I know what point you're
>making, I just disagree that it makes any kind of sense. More parallel
>*is* faster. If Computer A has twice as many processing cores as
>Computer B and each core has the same processing power, then Computer A
>is faster, pretty much by definition. That it will run some software no
>faster than Computer B would is true but irrelevant. This is especially
>true when you're talking about the Singularity, where the major event is
>the creation of superhuman AI, which is virtually certain to be a
>problem which parallelizes well.
The brain isn't so much like a Cray with difficult parallel computing
trying to come up with a single answer quickly. It's closer to the
Internet, with lots of inputs yelling at each other, and whichever
parts yelling loudest winning.
Right now, if you think singularity, it is a mistake to ignore
Internet trends.
Ahasuerus
[snip-snip]
> Who would be stupid enough to tie all his resources to a single point of failure?
Have you met any Evil Overlords lately?
Ahasuerus
> Aaron Bergman schrieb:
>
> > wrote:
> >> Yeah, it just means more things can be computed per given time interval.
> >> Which scarcely resembles "faster" at all.
>
> > If all you want to do is ray tracing, that's fine. If you want to
> > something more complicated (that requires random access to a large
> > amount of memory, say), you're going to have to work hard to make sure
> > all those cores aren't sitting idle waiting for data.
>
> The application in question was an A.I. - those are by definition a
> parallel thing. Its about processing input from various sources,
> crosslinking that massive amounts of prior experiences, exploring "what
> if"-scenarios. Even following imperative design patterns that all
ISFDB robots already check hundreds of data sources in parallel in
search of anything that looks like SF. Of course, they are well
behaved robots and would never turn on their creators...
ZnU
Mike Ash <mi...@mikeash.com> wrote:
> In article <znu-3CC20D.0...@Port80.Individual.NET>,
> ZnU <z...@fake.invalid> wrote:
>
> > Consult this handy chart:
> > http://en.wikipedia.org/wiki/File:PPTMooresLawai.jpg
> >
> > The silicon microprocessor is the fifth physical computing substrate
> > widely employed by humanity (not counting meat, anyway), and Moore's Law
> > appears to have been with us essentially since the beginning. There's no
> > particular reason to think it will end when silicon microprocessor
> > technology has run its course.
> >
> > (Actually, there *is* a bit of a logistic curve on that chart. It
> > appears to be bending... up.)
>
> Since the question at hand is what's been happening with computer speeds
> in the last 10 years or so, this chart, which ends in 2000, is pretty
> much pointless. I have no doubt that it would continue the trend if you
> added 2000-2009 to it, but as it stands that chart doesn't quite do the
> job.
Well, the purpose of the chart was not specifically to address the last ten
years, but to address the issue of whether Moore's Law would expire with the
silicon microprocessor.
Here's a chart that goes through 2008:
http://en.wikipedia.org/wiki/File:Transistor_Count_and_Moore%27s_Law_-_2008.svg
David DeLaney
>The brain isn't so much like a Cray with difficult parallel computing
>trying to come up with a single answer quickly. It's closer to the
>Internet, with lots of inputs yelling at each other, and whichever
>parts yelling loudest winning.
And every so often one of the yells from the inputs resonates temporarily
with part or all of the whole rest of the structure, and the whole thing
rings like a bell. Meanwhile, parts of it become self-contained whirlpools
of mutated meaning that are incomprehensible to the rest...
>Right now, if you think singularity, it is a mistake to ignore
>Internet trends.
Dave "one must be Aware Of All Internet Traditions" DeLaney
Bernard Peek
<z...@fake.invalid> writes
>The silicon microprocessor is the fifth physical computing substrate
>widely employed by humanity (not counting meat, anyway), and Moore's Law
>appears to have been with us essentially since the beginning. There's no
>particular reason to think it will end when silicon microprocessor
>technology has run its course.
Yes and no. There's no reason to suppose that we won't continue to
double the number of transistors per chip every 18 months for a few more
generations. However we are reaching the point where doubling the number
of transistors doesn't do much for the processing power of complete
systems.
What we need to do next is to run processors at faster clock-speeds,
which quite possibly involves a switch to exotic semiconductors. Even
that won't be enough in itself. What we will also need to do is to speed
up motherboards, and the limitation there is transmission speeds and
distances. Motherboards will need to shrink to perhaps a tenth of their
current size or less. That makes cooling (and lots more) a lot more
difficult.
The limit is what was once referred to as a "hairy golfball" which is a
spherical computer with connecting wires coming out in all directions.
--
Bernard Peek
Default User
> Of course, they are well
> behaved robots and would never turn on their creators...
Those are not the fun kind.
Brian
--
Day 263 of the "no grouchy usenet posts" project
W. Citoan
> In article <mike-3E2E2E.1...@news.eternal-september.org>,
> Mike Ash <mi...@mikeash.com> wrote:
>
> > In article <znu-3CC20D.0...@Port80.Individual.NET>,
> > ZnU <z...@fake.invalid> wrote:
> >
> > > Consult this handy chart:
> > > http://en.wikipedia.org/wiki/File:PPTMooresLawai.jpg
> > >
> > > The silicon microprocessor is the fifth physical computing substrate
> > > widely employed by humanity (not counting meat, anyway), and Moore's Law
> > > appears to have been with us essentially since the beginning. There's no
> > > particular reason to think it will end when silicon microprocessor
> > > technology has run its course.
> > >
> > > (Actually, there *is* a bit of a logistic curve on that chart. It
> > > appears to be bending... up.)
> >
> > Since the question at hand is what's been happening with computer speeds
> > in the last 10 years or so, this chart, which ends in 2000, is pretty
> > much pointless. I have no doubt that it would continue the trend if you
> > added 2000-2009 to it, but as it stands that chart doesn't quite do the
> > job.
>
> Well, the purpose of the chart was not specifically to address the last ten
> years, but to address the issue of whether Moore's Law would expire with the
> silicon microprocessor.
>
> Here's a chart that goes through 2008:
> http://en.wikipedia.org/wiki/File:Transistor_Count_and_Moore%27s_Law_-_2008.svg
That's a chart of transistor counts & not speed. While Moore's Law is
often cited in discussions of speed, Moore's Law states transistor
counts will double; not that speed will double. While they are related,
they are not the same thing.
- W. Citoan
--
This book is dedicated to my brilliant and beautiful wife without whom I
would be nothing. She always comforts and consoles, never complains or
interferes, asks nothing, and endures all. She also writes my dedications.
-- Albert Malvino
Tim Little
> The limit is what was once referred to as a "hairy golfball" which is a
> spherical computer with connecting wires coming out in all directions.
Except for the problem that spheres are a terrible shape for removing
heat from the interior.
- Tim
ZnU
"W. Citoan" <wci...@NOSPAM-yahoo.com> wrote:
As others have pointed out, they effectively *are* the same thing for
sufficiently parallelizable processes. Which we basically know that
intelligence is, because that appears to be how the brain works.
ZnU
wrote:
> In message <znu-3CC20D.0...@Port80.Individual.NET>, ZnU
> <z...@fake.invalid> writes
>
>
> >The silicon microprocessor is the fifth physical computing substrate
> >widely employed by humanity (not counting meat, anyway), and Moore's Law
> >appears to have been with us essentially since the beginning. There's no
> >particular reason to think it will end when silicon microprocessor
> >technology has run its course.
>
> Yes and no. There's no reason to suppose that we won't continue to
> double the number of transistors per chip every 18 months for a few more
> generations. However we are reaching the point where doubling the number
> of transistors doesn't do much for the processing power of complete
> systems.
It doesn't do much for the execution speeds of unmodified software
written using today's common methods.
That doesn't have much to do with the long-term prospects of AI.
> What we need to do next is to run processors at faster clock-speeds,
> which quite possibly involves a switch to exotic semiconductors. Even
> that won't be enough in itself. What we will also need to do is to speed
> up motherboards, and the limitation there is transmission speeds and
> distances. Motherboards will need to shrink to perhaps a tenth of their
> current size or less. That makes cooling (and lots more) a lot more
> difficult.
>
> The limit is what was once referred to as a "hairy golfball" which is a
> spherical computer with connecting wires coming out in all directions.
If you want to know where commodity computing is going, look at
supercomputers, where nobody even really seriously bothers to consider
alternatives to parallel architectures anymore. Commodity computers will
because increasingly parallel, and NUMA will probably be necessary at
some point as well. Yes, this makes writing software harder, at least
with current tools. Yes, it doesn't work for every single algorithm. But
tools will improve, and it works for a very large number of useful
algorithms.
Wayne Throop
:: double the number of transistors per chip every 18 months for a few
:: more generations. However we are reaching the point where doubling
:: the number of transistors doesn't do much for the processing power of
:: complete systems.
: It doesn't do much for the execution speeds of unmodified software
: written using today's common methods. That doesn't have much to do
: with the long-term prospects of AI.
Nor imo even the short to medium term functionality of personal computers
without explicitly "AI" going on, really.
Let's assume for a moment that from now on, clock speed never increases,
and all those transistors go into doubling the number of cores and
memory and such every year or two. So in ten years, a typical computer
might have a few thousand cores. Do you think a computer then will
be doing no more computation per unit time than a computer does now?
I suspect it'll be doing *lots* more, and a lot of it will be in terms
of autonomous agents acting on various long-term goals.
And that's even without radically changing software style, but rather,
just running locally processes that right now take compute farms; things
like rendering images, searches for data, pro-active computation of things
the software suspects the user might want in the near future, etc, etc.
Consider that right now, the machine I'm using has only a single core
(though my laptop does hyperhthreading...). And there are several tasks I
use it for that end up splitting its cpu up into three or four relatively
independent tasks, each cpu intensive. Displaying a VOD training film
from work means the X server, the movie player, encryption/decryption
of vpn links, and a few miscelaneous things, are all sucking the CPU
right out of it. All those things could, with some benefit, be put onto
different cores. And if the file indexer was at work, or I was mucking
about with moving things around on the local net at the same time (eg,
doing backups, etc), still more would be useful, and would directly
improve responsiveness of the system.
So even changing relatively little, I could pine for the fjords of a
few years from now, where 8 cores are cheap and common tech. And if
I were running google engines locally or graphics rendering of virtual
whatsis or lots of other things I don't do now because they'd cause the
system to drag, I'd pine fo even *more* cores, without really changing
programming style.
And I'm horribly behind the times on finding uses for my computer.
Certainly, the corporate computing environment at work is just *loving*
the fact that they can throw more cores at things; on that scale, things
really are already divided up into a zillion tasks that need to run at
the same time... and if they can run them all on a rack instead of a
room full of servers, they are very happy. And they normally find that
user demand for services keeps up when any expansions they offer.
So. Anyways. I think throwing cores at people will, even in the
short term and without AI or radical reorganization of software,
make people almost as happy as throwing hertz at them used to do.
W. Citoan
> In article <slrnhe4amd....@wcitoan-via.eternal-september.org>,
> "W. Citoan" <wci...@NOSPAM-yahoo.com> wrote:
>
> > ZnU wrote:
> > >
> > > Well, the purpose of the chart was not specifically to address the
> > > last ten years, but to address the issue of whether Moore's Law
> > > would expire with the silicon microprocessor.
> > >
> > > Here's a chart that goes through 2008:
> > > http://en.wikipedia.org/wiki/File:Transistor_Count_and_Moore%27s_La
> > > w_-_2008 .svg
> >
> > That's a chart of transistor counts & not speed. While Moore's Law
> > is often cited in discussions of speed, Moore's Law states transistor
> > counts will double; not that speed will double. While they are
> > related, they are not the same thing.
>
> As others have pointed out, they effectively *are* the same thing for
> sufficiently parallelizable processes.
No, others have not pointed that out. Others have talked about the
implications of parallelization, but that is entirely different than
the number of transistors on a single die. Moore's Law is not about
parallelization.
- W. Citoan
--
Happy is the man that findeth wisdom, and the man the getteth
understanding.
-- Proverbs (Bible)
Mike Ash
ZnU <z...@fake.invalid> wrote:
> If you want to know where commodity computing is going, look at
> supercomputers, where nobody even really seriously bothers to consider
> alternatives to parallel architectures anymore. Commodity computers will
> because increasingly parallel, and NUMA will probably be necessary at
> some point as well. Yes, this makes writing software harder, at least
> with current tools. Yes, it doesn't work for every single algorithm. But
> tools will improve, and it works for a very large number of useful
> algorithms.
And when was the last time a serious supercomputer was built that WASN'T
parallel? Even the 1982 Cray X-MP could be configured with up to four
processors. For Crays you have to go back to the Cray 1 to get one that
wasn't designed for multiprocessing, and even the Cray 1 was heavily
designed around vector processing, which is just a different kind of
parallelism.
In short: you're right, this whole trend toward parallelism is nothing
new.
Mike Ash
"W. Citoan" <wci...@NOSPAM-yahoo.com> wrote:
> ZnU wrote:
> > In article <slrnhe4amd....@wcitoan-via.eternal-september.org>,
> > "W. Citoan" <wci...@NOSPAM-yahoo.com> wrote:
> >
> > > ZnU wrote:
> > > >
> > > > Well, the purpose of the chart was not specifically to address the
> > > > last ten years, but to address the issue of whether Moore's Law
> > > > would expire with the silicon microprocessor.
> > > >
> > > > Here's a chart that goes through 2008:
> > > > http://en.wikipedia.org/wiki/File:Transistor_Count_and_Moore%27s_La
> > > > w_-_2008 .svg
> > >
> > > That's a chart of transistor counts & not speed. While Moore's Law
> > > is often cited in discussions of speed, Moore's Law states transistor
> > > counts will double; not that speed will double. While they are
> > > related, they are not the same thing.
> >
> > As others have pointed out, they effectively *are* the same thing for
> > sufficiently parallelizable processes.
>
> No, others have not pointed that out. Others have talked about the
> implications of parallelization, but that is entirely different than
> the number of transistors on a single die. Moore's Law is not about
> parallelization.
Except that doubling your transistor count lets you double the number of
processor cores you can put on the same chip, all else being equal
(assuming you can remove the heat). So yes, actually, it is pretty much
about parallelization. The only reason you didn't see lots of multi-core
processors in 1990 was because the software wasn't up to it and chip
designers still had a lot of tricks to use those extra transistors for
greater single-core speeds instead. (And a lot of those tricks involved
implicit parallelization being performed at the hardware level.)
DouhetSukd
>
> What we need to do next is to run processors at faster clock-speeds,
> which quite possibly involves a switch to exotic semiconductors. Even
> that won't be enough in itself. What we will also need to do is to speed
> up motherboards, and the limitation there is transmission speeds and
> distances. Motherboards will need to shrink to perhaps a tenth of their
> current size or less. That makes cooling (and lots more) a lot more
> difficult.
>
I'm afraid I have to disagree. The limitation IMHO is not so much
bottle-necking speeds as not being sure what to do with them.
If you look at AI savvy-layman-level wisdom, or what the AI
researchers tell us, they've had great expectations for... 40-50 years
now?
In that time, processing speed has gone by a huge factor and we are
learning that we are not that much closer. Yup, we've finally beaten
Kasparov, in a very very specialized application of computing logic,
at huge cost. Would 10, 100, 1000 times the speed suddenly catapult
us to true AI success? We've started to understand how the brain
works with cross-pollination with neural network processing. In many
fields, specialized AI _is_ very helpful and I expect it to be become
ever more so. True, we have data-mining, optical recognition of
shapes, driven by military research among other things. We have ever
smarter computer opponents in computer games. The military still
flies Reapers remotely, not by onboard logic (though the Army has
fewer mishaps with their UAVs by letting them land themselves).
But we don't seem so very close to a general-purpose self-learning
algorithm that evolves on its own. Let alone self-aware. I concur
that the Internet is to be watched - perhaps all that available
digital information can be used as input to learning systems, now that
it is there for the taking.
And, singularity-wise, wisdom may come in in little unexpected
packets. Say an specialized AI that could figure out how molecules
work so that it could design drugs better than humans. That would be
a huge advance, a huge societal shift and would trigger research in
areas that would appear to potentially benefit. But why would it not
work with massively parallel systems? Some people have had programs
design integrated circuits, using genetic algorithms.
Some of my smarter computer brethren frequently advise against using
threading at the application level, because it is usually so hard to
keep track of and debug. I believe part of the gap is acquiring
better toolsets, languages, and mental models to deal with
parallelism. Not pure mobo speed.
Besides I also disagree with the OPs statements. Marketing _has_
shifted away from GHz but that was because the Intel P4 CPUs
demonstrated the silliness of that metric. And because modern CPUs
run much of what a user might want without breaking a sweat, despite
Vista's best attempts. A better metric might the top 500 computer
lists: 2001 numbers look like 2009 numbers, until you see it uses
GFlops rather than TFlops as units.
Tim Little
> The only reason you didn't see lots of multi-core processors in 1990
> was because the software wasn't up to it and chip designers still
> had a lot of tricks to use those extra transistors for greater
> single-core speeds instead. (And a lot of those tricks involved
> implicit parallelization being performed at the hardware level.)
Yes, even pipelining is a form of parallelization. Speculative
execution, register renaming, ALU widening, vector processing, and
hardware multithreading even more directly so. The big difference
between these and multiple cores is that these operations are very,
very tightly coupled while multicore CPUs more commonly operate on
completely independent instruction streams (and even at different
clock rates) with synchronization being expensive.
It seems to me that many operations in brains correspond more to
closely coupled parallelism than to independence. Still, I don't
doubt that the relevant aspects of its behaviour can be sufficiently
well modelled with loosely-coupled parallel components.
- Tim
Wayne Throop
: And, singularity-wise, wisdom may come in in little unexpected
: packets. Say an specialized AI that could figure out how molecules
: work so that it could design drugs better than humans. That would be
: a huge advance, a huge societal shift and would trigger research in
: areas that would appear to potentially benefit. But why would it not
: work with massively parallel systems? Some people have had programs
: design integrated circuits, using genetic algorithms.
Interesting thought. It wouldn't even need to be a *specialized* AI;
Eurisko, a heristics engine, did various interesting things, such as
design integrated circuits, beat human players at war games, and
play chess (though not better than a human or a specialized program).
It was also the kind of process that was highly parallelizable, and
was let run in the background over long periods of time thinking
deep thoughts, soaking up CPU cycles.
However, of course, it's early promise didn't really pay off, and remnants
of it are in the "Cyc" project, but nothing really amazing that I know of
came of it.
The stuff I read about it said that the ideal problem for Eurisko was one
with a huge search space which humans hadn't explored much yet. Molecular
biology might be one such... but of course specialized programs like
protein folding algorithms and various processing of DNA sequences seem
to dominate the field, not any general-purpose reasoning engine. Plus,
of course, Cyc is focussed on capturing "common sense", not abstract
engineering problems.
Eh. Anyhows. http://en.wikipedia.org/wiki/Eurisko
http://en.wikipedia.org/wiki/Cyc
ZnU
wrote:
> :: Yes and no. There's no reason to suppose that we won't continue to
> :: double the number of transistors per chip every 18 months for a few
> :: more generations. However we are reaching the point where doubling
> :: the number of transistors doesn't do much for the processing power of
> :: complete systems.
>
> : It doesn't do much for the execution speeds of unmodified software
> : written using today's common methods. That doesn't have much to do
> : with the long-term prospects of AI.
>
> Nor imo even the short to medium term functionality of personal computers
> without explicitly "AI" going on, really.
>
> Let's assume for a moment that from now on, clock speed never increases,
> and all those transistors go into doubling the number of cores and
> memory and such every year or two. So in ten years, a typical computer
> might have a few thousand cores. Do you think a computer then will
> be doing no more computation per unit time than a computer does now?
> I suspect it'll be doing *lots* more, and a lot of it will be in terms
> of autonomous agents acting on various long-term goals.
Not to mention all the routine tasks that already consist of performing
the same operations on large data sets with no real serialization
required. For instance, most processing that you'd want to do with video
(decoding, encoding, adding filters, compositing, etc.) can be
parallelized almost trivially. Have 8 cores? Process 8 frames at once.
Have 1024 cores? Process 1024.
And the developer tools to make this stuff easier *are* starting to fall
into place. Sure, it will never be as simple as writing single-threaded
code was -- not without a major paradigm shift in how programming works,
anyway -- but, well, it's a lot more complicated to build a modern car
than a 1950 model as well. Why would computer software be different?
> And that's even without radically changing software style, but rather,
> just running locally processes that right now take compute farms; things
> like rendering images, searches for data, pro-active computation of things
> the software suspects the user might want in the near future, etc, etc.
>
> Consider that right now, the machine I'm using has only a single core
> (though my laptop does hyperhthreading...). And there are several tasks I
> use it for that end up splitting its cpu up into three or four relatively
> independent tasks, each cpu intensive. Displaying a VOD training film
> from work means the X server, the movie player, encryption/decryption
> of vpn links, and a few miscelaneous things, are all sucking the CPU
> right out of it. All those things could, with some benefit, be put onto
> different cores. And if the file indexer was at work, or I was mucking
> about with moving things around on the local net at the same time (eg,
> doing backups, etc), still more would be useful, and would directly
> improve responsiveness of the system.
>
> So even changing relatively little, I could pine for the fjords of a
> few years from now, where 8 cores are cheap and common tech. And if
> I were running google engines locally or graphics rendering of virtual
> whatsis or lots of other things I don't do now because they'd cause the
> system to drag, I'd pine fo even *more* cores, without really changing
> programming style.
Yup. I'm in pro video. We have a couple of 8 core machines at the
office. One of them is running three different rendering/encoding jobs
overnight, right now. I'll take as many cores as I can get, thanks.
> And I'm horribly behind the times on finding uses for my computer.
>
> Certainly, the corporate computing environment at work is just *loving*
> the fact that they can throw more cores at things; on that scale, things
> really are already divided up into a zillion tasks that need to run at
> the same time... and if they can run them all on a rack instead of a
> room full of servers, they are very happy. And they normally find that
> user demand for services keeps up when any expansions they offer.
>
> So. Anyways. I think throwing cores at people will, even in the
> short term and without AI or radical reorganization of software,
> make people almost as happy as throwing hertz at them used to do.
Especially when combined with throwing huge GPU performance increases at
them. Nvidia's predicting a 570x performance increase over the next six
years. (Of course the lines between GPUs and CPUs will probably be
mostly gone in a decade.)
> Wayne Throop thr...@sheol.org http://sheol.org/throopw
--
Wayne Throop
: And when was the last time a serious supercomputer was built that WASN'T
: parallel? [...] this whole trend toward parallelism is nothing new.
Not new, and already been carried to quite an extreme.
The top machine on the "top 500" computer list has 0.13 *million* cores.
I note that in "across realtime", one of the computing devices mentioned
wrt Tunc Blumenthal's equipment, involved compting cores no more powerful
than an early intel x86 chip (though, I think, faster clock rate).
Except they were placed something one per square angstrom across the
entire surface of Tunc's craft.
Tim Little
> I'm afraid I have to disagree. The limitation IMHO is not so much
> bottle-necking speeds as not being sure what to do with them.
I'd say the opposite: doubling clock speed is always more useful than
doubling processor cores. At worst, it can emulate two processor
cores within arbitrary precision of the same time. At best, it can
outperform them by 100%. If we could run a particular class of CPU
core at 4 GHz instead of producing two 2 GHz cores at the same costs,
it would always be better and the multi-core possibility would be
ignored.
We have multi-core chips only because we've run into a raw speed
barrier.
> In that time, processing speed has gone by a huge factor and we are
> learning that we are not that much closer. Yup, we've finally
> beaten Kasparov, in a very very specialized application of computing
> logic, at huge cost.
We're still far below the computational capacity and power of a human
brain. Perhaps what we are finding is that some AI problems
inherently require a decent fraction of that capacity.
Besides, next year we'll beat Kasparov much more cheaply. What's
more: now that we've calibrated the limits of human chess-playing
ability, we have an automated benchmark against which we can construct
better and more efficient chess computers. Eventually it may serve as
one of many data points for the adaptability of systems that were not
designed to play chess, but had to learn.
> But we don't seem so very close to a general-purpose self-learning
> algorithm that evolves on its own.
Perhaps we do, but it is both crippled by insufficient data capacity
and so slow on current systems that we don't recognise it as
functioning before moving on to something more promising. If a
general-purpose learning algorithm performed exactly as well as a
newborn baby but 1/10th the speed, the approach would be abandoned.
Now that I think more, it would probably be abandoned even if it
learned at the same rate: "Test run 256. Software agent with six
months environmental learning experience was placed at entrance to
maze. It entered the maze, made two turns, then stopped and emitted
semiperiodic signals carrying no apparent information content until it
was switched off."
> Some of my smarter computer brethren frequently advise against using
> threading at the application level, because it is usually so hard to
> keep track of and debug. I believe part of the gap is acquiring
> better toolsets, languages, and mental models to deal with
> parallelism. Not pure mobo speed.
Pure speed avoids the need to design around parallelism. It worked
for a while, but increasing the speed has now reached the point where
it is more costly than dealing with parallelism.
- Tim
ZnU
"W. Citoan" <wci...@NOSPAM-yahoo.com> wrote:
> ZnU wrote:
> > In article <slrnhe4amd....@wcitoan-via.eternal-september.org>,
> > "W. Citoan" <wci...@NOSPAM-yahoo.com> wrote:
> >
> > > ZnU wrote:
> > > >
> > > > Well, the purpose of the chart was not specifically to address the
> > > > last ten years, but to address the issue of whether Moore's Law
> > > > would expire with the silicon microprocessor.
> > > >
> > > > Here's a chart that goes through 2008:
> > > > http://en.wikipedia.org/wiki/File:Transistor_Count_and_Moore%27s_La
> > > > w_-_2008 .svg
> > >
> > > That's a chart of transistor counts & not speed. While Moore's Law
> > > is often cited in discussions of speed, Moore's Law states transistor
> > > counts will double; not that speed will double. While they are
> > > related, they are not the same thing.
> >
> > As others have pointed out, they effectively *are* the same thing for
> > sufficiently parallelizable processes.
>
> No, others have not pointed that out. Others have talked about the
> implications of parallelization, but that is entirely different than
> the number of transistors on a single die. Moore's Law is not about
> parallelization.
Not directly, but the implication is obvious. If transistor density
keeps going up, but clock speed stalls out, the solution is on-chip
parallelism. Which is exactly what we've seen over the last few years,
and what CPU and GPU vendors seem to be planning a lot more of in the
future.
ZnU
<36ac4857-eabe-4531...@f20g2000prn.googlegroups.com>,
DouhetSukd <douhe...@gmail.com> wrote:
We still don't really have the hardware for it. Some believed that we
could implement AI from first principles using various sorts of rule or
logic-based systems, but that approach has been a dead end for
generalized intelligence. Our best bet at this point is brute force
simulation of the brain, which depending on who you listen to requires
something between 10 and 100,000 petaflops. The fastest supercomputers
are just entering that range within the next year or two, and we'll be
over the top around 2025 or 2030, if Moore's Law keeps up.
Henry Markram, director of the Blue Brain Project, commented at TED that
he sees simulations of the entire human brain being plausible within a
decade, and he'd probably know if anyone would. Blue Brain currently has
a functional simulation of a neocortical column of a rat; baring
something unforeseen, it's just a matter of scale to get from there to
full human brain simulations.
> I concur that the Internet is to be watched - perhaps all that
> available digital information can be used as input to learning
> systems, now that it is there for the taking.
>
> And, singularity-wise, wisdom may come in in little unexpected
> packets. Say an specialized AI that could figure out how molecules
> work so that it could design drugs better than humans. That would be
> a huge advance, a huge societal shift and would trigger research in
> areas that would appear to potentially benefit. But why would it not
> work with massively parallel systems? Some people have had programs
> design integrated circuits, using genetic algorithms.
>
> Some of my smarter computer brethren frequently advise against using
> threading at the application level, because it is usually so hard to
> keep track of and debug. I believe part of the gap is acquiring
> better toolsets, languages, and mental models to deal with
> parallelism. Not pure mobo speed.
>
> Besides I also disagree with the OPs statements. Marketing _has_
> shifted away from GHz but that was because the Intel P4 CPUs
> demonstrated the silliness of that metric. And because modern CPUs
> run much of what a user might want without breaking a sweat, despite
> Vista's best attempts. A better metric might the top 500 computer
> lists: 2001 numbers look like 2009 numbers, until you see it uses
> GFlops rather than TFlops as units.
1996 saw the world's first teraflop supercomputer. IBM's Sequoia, coming
online at the US DoE in 2011, will be 20 petaflops, 20,000x as fast.
There's also supposedly an exaflop machine (50 times as fast again)
being contemplated to process data from the Square Kilometer Array.
Mike Ash
<36ac4857-eabe-4531...@f20g2000prn.googlegroups.com>,
DouhetSukd <douhe...@gmail.com> wrote:
> Some of my smarter computer brethren frequently advise against using
> threading at the application level, because it is usually so hard to
> keep track of and debug. I believe part of the gap is acquiring
> better toolsets, languages, and mental models to deal with
> parallelism. Not pure mobo speed.
It's interesting how people are always talking about how difficult
multi-threaded programming is (and justifiably so) but completely gloss
over all the difficulties involved in making new and better chips.
Each new advance in feature size reduction takes years of intense
research and billions of dollars to achieve. This is a hugely difficult
undertaking, and it's never certain to succeed. Yet it always has, and
it happens in a way that's mostly invisible to us, so we tend to take it
for granted.
You're completely right that the tools and techniques need significant
research to take advantage of tomorrow's mulit-core monsters. But
tomorrow's multi-core monsters also need a great deal of time and money
to come about. All of this stuff involves lots and lots of really smart
people dedicated to tackling the problem, and multithreaded software is
in no way unique in that respect.
Bernard Peek
Ash <mi...@mikeash.com> writes
>
>Except that doubling your transistor count lets you double the number of
>processor cores you can put on the same chip, all else being equal
>(assuming you can remove the heat). So yes, actually, it is pretty much
>about parallelization. The only reason you didn't see lots of multi-core
>processors in 1990 was because the software wasn't up to it and chip
>designers still had a lot of tricks to use those extra transistors for
>greater single-core speeds instead. (And a lot of those tricks involved
>implicit parallelization being performed at the hardware level.)
Cf: Transputer
--
Bernard Peek
Bernard Peek
writes
>:: Yes and no. There's no reason to suppose that we won't continue to
>:: double the number of transistors per chip every 18 months for a few
>:: more generations. However we are reaching the point where doubling
>:: the number of transistors doesn't do much for the processing power of
>:: complete systems.
>
>: It doesn't do much for the execution speeds of unmodified software
>: written using today's common methods. That doesn't have much to do
>: with the long-term prospects of AI.
>
>Nor imo even the short to medium term functionality of personal computers
>without explicitly "AI" going on, really.
That's where we disagree. I'm sure that multiple processors will improve
the performance of home computers but I'm not expecting a vast
performance increase, maybe a factor of ten or so for massively parallel
systems written by optimising compilers and using transistors a tenth of
their current size. Possibly not even that much. All of that increased
performance will be absorbed by the software, leaving machines
subjectively at about the same speed as they are now.
There are two problems that I can see. The first is that there is an
overhead in the processes that coordinate the work of multiple
processors. As the number of processors goes up that overhead increases
and it will increase faster than the rate of increase in the number of
cores.
The second problem is that with multiple cores communications paths
between cores start to become bottlenecks.
For both effects processing power increases linearly with the number of
processors, but overheads increase geometrically or faster. We aren't
very far along that road yet so neither effect is much of a problem.
--
Bernard Peek
DouhetSukd
> In article
> <36ac4857-eabe-4531-893b-17fb157a0...@f20g2000prn.googlegroups.com>,
Depends on your perspective. I am a business apps programmer,
specialized in databases. Most of the time, something I write is
counted in man-days, maybe man-months, me being the only coder. There
just isn't the time budget to cover all the bases by the very thorough
design and testing that the additional complexity of threads would
require. Not to mention debugging and increased support
requirements.
Sure, given a big team and a long time, people can write complex multi-
threaded stuff. After all networks, OSs, databases, better be using
threads or multiple interacting processes. But that work is more akin
to chip design in terms of methodology, time, budget and team volumes
anyway.
The productivity tradeoff is just not justifiable for applications of
many types, prices and distribution volumes.That's why I said
"application level". Just because you have a tool doesn't mean you
should use it. Same consideration applies to _complex_ object
oriented programming - OO can solve many problems, but sometimes the
end program is over-designed for the problem it was trying to solve.
Mind you, I also think C++ is nearly useless for non-system
programming and chuckle at business programmers who claim C++ is
great. And I am on record for calling Java the COBOL of the 21st
century. So I'm lazy and prejudiced.
Robert Martinu
> It seems to me that many operations in brains correspond more to
> closely coupled parallelism than to independence.
Think of all that processing that happens on lower levels, either as
part of the vegetative system or in the subconsciousness.
Or the visual system, lots of preprocessing within or close to the
retina, another midlevel layer that translates visual impressions into a
logical model your mind works with. There isn't that much feedback or
tight linking as you'll find out if you try to disbelieve optical
illusions. That old woman/young lady-picture comes to mind.
Also consider synaestetics or similar phenomenas. They give a fair idea
on what levels our senses come together for the first time - and to what
degree.
Robert Martinu
> On Oct 23, 11:20 am, Robert Martinu <inva...@invlid.invalid> wrote:
> [snip-snip]
>> Who would be stupid enough to tie all his resources to a single point of failure?
>
> Have you met any Evil Overlords lately?
For them its part of their contract.
Union will give them hell if they don't incorporate the plot relevant
design flaws - sad sort of job actually, with that much executive meddling.
Wayne Throop
: I'm sure that multiple processors will improve the performance of home
: computers but I'm not expecting a vast performance increase, maybe a
: factor of ten or so for massively parallel systems written by
: optimising compilers and using transistors a tenth of their current
: size. Possibly not even that much. All of that increased performance
: will be absorbed by the software, leaving machines subjectively at
: about the same speed as they are now.
I'm not sure that's really disagreement. Sounds much like what I'd expect.
Today, for many tasks machines are nigh-instantaneous. I expect that'll
remain true but it'll be possible to do many more nigh-instantaneous
things at once, and it'll be feasible to do speculative computation of
things that *might* be useful in the near future, etc, etc. That might
extend things another order of magnitude.
Now, you can say that that's a case of the raw capability being
"absorbed by software"; that'd render the above compatible with my
expectations. But I don't think it'll be as simple as that.
There really will be value added by all that "bloat"ed computations
going on in this scenario. Possibly even more than is provided by
today's bloat.
I mean... people said the same thing about more hertz then as is said
of more cores now. The machine is already fast enough, I was running X
windows on a machine with a few megahertz, one megabyte and 1 megapixel,
and the only thing that'll be added is useless frippery like color,
transparency, filled move, photo backgrounds and desktops, scaling,
antialiased fonts... none of which are really *needed*. But... it also
allowed such things as video processing, real-time ray tracing, etc, etc,
etc, etc. All of which you can argue are just useless bloat, compared
to what you really need done which is email and simple word processing.
Or consider how much cpu horsepower it took to render web pages in the
late '90s com pared to the late '00s. Sure, one can grumble that the
silly web pages are just bloat and don't need to be anything more than
text, gifs, and mayb jpegs... but... even 1980s machines would choke
on many images, or much text formatting.
Just which things are bloat and which are useful features? It's all a
matter of perspective.
Of course, like I said, that only extends to an order of magnitude
or *maybe* two before it becomes irrelevant. But that's six or more
years off, so by then, I expect the next level of bloat will appear
on the horizon to soak up another order of magnitude after that.
"The boy's suit I designed to withstand enormous friction
without heating up or wearing out, a useful feature."
--- Edna Mode (and all that is just excess bloat
compared to good old dynamesh, if you ask me)
Mike Ash
wrote:
> In message <12563...@sheol.org>, Wayne Throop <thr...@sheol.org>
You seem to be talking as though these are all new and unexplored areas,
but it's not true by a long shot.
Intel Paragon, launched in 1992, could be configured with up to 2048
processors. Supercomputers have been massively parallel for a long, long
time now, and they don't get overwhelmed by geometrically increasing
overhead. Yes, they don't see a best-case speedup very often, and some
problems are not well suited to them at all, but a LOT of useful
computation gets done too.
There are some types of functionality in a personal computer that would
benefit greatly from having faster single-threaded CPUs but which can't
take advantage of parallel computation. But there are also a LOT of
things that a PC can do usefully that DO take advantage of parallel
computation. Just because not everything can benefit doesn't mean that
what's left over isn't significant.
Achieving performance on massively parallel machines is by no means a
solved problem, but neither is it a mysterious unknown world that nobody
has explored yet. It is possible, even feasible, and not even all that
hard for many problems.
Wayne Throop
: For both effects processing power increases linearly with the number of
: very far along that road yet so neither effect is much of a problem.
Except of course for supercomputers, which are quite a bit further
along that road and have as many as 0.13 million cores. And more are
planned for the immediate future in the million-core range. Meanwhile,
corporations are swapping in racks full of 8-core machines with mere
gigabit ethernet interconnect as generic compute servers; a lab full
can have many many thousands of cores, and all working on a suite of
related problems. And then there's google.
On an unrelated note, I'm typing this in on a machine many times faster
than a Cray I. Well actually, I'm typing it in on two machines each of
which are many times faster than a Cray I; one to handle the display,
keyboard, and mouse, and the other to handle network connections, file
system, heavier compute tasks, web page renders, and misc other stuff.
Um... ignoring the two routers and a cable modem still inside this house
before you get to my ISP. All of which are more powerful computers than
those provided by my first employers in the computer biz in the late 70s
and early 80s. All of which, I note, is pure bloat from the perspective
of the mid-80s, and nigh-miraculous from the perspective of the mid-70s.
In the late 70s, with a mainframe which was supporting a whole university
campus worth of student and faculty computation, and on off-hours and
on the weekend, I could type "enter" on the command to run just about
any program I wanted to, and have the printer start up with the results
in a quarter of a second. What in the world would I ever need anything
faster than that? Surely, if everybody had their own computer that fast,
nothing more would be needed. Well... I now have several computers much
faster, doing myriad nigh-invisible tasks in the background that would
have brought that system to its knees, while in the "idle load" state (ie,
less than 1 percent cpu utilization), and don't find it excessive; and
I'm not even really on any of the cutting edges of computer use at all.
So. History leads me to continue to have faith that things will be
found to keep lots of cores busy, even on lowly personal computers.
And that some people will lament that it's all bloat, and others will
find said bloat to actually be novel, useful capabilities. So I rather
expect in ten or 20 years time for people to have PCs which are to
today's supercomputers what my compting setup is to a Cray. And for
them to count what it's doing as mostly-useful, with a bit of bloat
compared to the previous generation of PCs. Everybody will have
their own compzilla.
History shows again and again
How nature points up the folly of men
Godzilla!
--- Blue Oyster Cult
Wayne Throop
: Depends on your perspective. I am a business apps programmer,
: specialized in databases. Most of the time, something I write is
: counted in man-days, maybe man-months, me being the only coder. There
: just isn't the time budget to cover all the bases by the very thorough
: design and testing that the additional complexity of threads would
: require. Not to mention debugging and increased support
: requirements.
Sure, but you often program in inherrently paralizable operations at
that level, don't you? Search a database for this, find records such
that that, put things on the display in such-and-such format, etc, etc.
All of which occur in processes that have threads. I mean, something
as simple as coding a web page nowdays runs in an environment that
has threads underlying it (or could, if it didn't), ie, a web browser.
One of the things I specialize in is makefile rules. And you don't
worry about threads when writing those (beyond a few rules of basic
hygiene), yet when make runs, it sure finds lots of things to do at once
on any task beyond the trivial. I certainly wouldn't want to explicitly
code the threads for that, yeeeeesh.
So, just as nobody back in the day spent the time and trouble to
actually do the finicky code to do numerical floating-point operations
including trig, log, etc, etc, etc, but they still wrote computer programs
that called canned routines that did these things, today programmers don't
really write things in terms of threads very often, but they call libraries
and services that *do* use threads extensively.
The higher-level the language, the more likely that parallelism can be
used in implementing its primitives. Even something as mundane as a
unix pipeline, or a spreadsheet, is often drastically parallelizable.
And of course query languages and reasoning engines and ray tracers and
on and on and on are becoming *more* common rather than less.
Wayne Throop
: Intel Paragon, launched in 1992, could be configured with up to 2048
: time now, and they don't get overwhelmed by geometrically increasing
: overhead. Yes, they don't see a best-case speedup very often, and some
: problems are not well suited to them at all, but a LOT of useful
: computation gets done too.
Well, for that matter, it's not as if just cranking up the CPU clock
got you best-case speeups either. And this was true even before things
like pipelining and/or superscalar meant that part of that was due to
parallelization issues; waiting for memory cycles, or cache fills, or for
secondary storage, meant that nothing was ever as CPU-bound and linearly
scalable as might have been hoped. So again, not exactly a new problem,
and not a problem that just cranking faster CPUs helps, any more
than just throwing *more* CPUs at it would help.
In CPU-intensive tasks I'm concerned with though (compilation of large
software systems), it's often the case that throwing twice the cores at
the problem will get you very nearly twice the throughput; 1.8 or 1.9
times not being uncommon. Not, of course, universally true, but still.
There are *lots* of tasks that throwing cores at *does* help, and there
are more of these on a PC than you might at first think, imo; and now
that the cores tend to be there, I expect people to exploit them more
and more, so more task will be *able* to benefit from them over time.
Erik Max Francis
> On an unrelated note, I'm typing this in on a machine many times faster
> than a Cray I. Well actually, I'm typing it in on two machines each of
> which are many times faster than a Cray I; one to handle the display,
> keyboard, and mouse, and the other to handle network connections, file
> system, heavier compute tasks, web page renders, and misc other stuff.
> Um... ignoring the two routers and a cable modem still inside this house
> before you get to my ISP. All of which are more powerful computers than
> those provided by my first employers in the computer biz in the late 70s
> and early 80s. All of which, I note, is pure bloat from the perspective
> of the mid-80s, and nigh-miraculous from the perspective of the mid-70s.
Note that not all "code bloat" is the same. It used to be the case that
you had to squeeze every last processor cycle and byte of memory dry to
get what you want done. The lifting of that cap has allowed a lot of
sloppy programming, but at the same time it's also allowed programmers
to spend their time more valuable ways -- for example, the proliferation
of high-level languages which would simply not have been feasible to use
on previous generations of machines.
--
Erik Max Francis && m...@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 18 N 121 57 W && AIM/Y!M/Skype erikmaxfrancis
Diplomacy and defense are not substitutes for one another. Either
alone would fail. -- John F. Kennedy, 1917-1963
Erik Max Francis
> : DouhetSukd <douhe...@gmail.com>
> : Depends on your perspective. I am a business apps programmer,
> : specialized in databases. Most of the time, something I write is
> : counted in man-days, maybe man-months, me being the only coder. There
> : just isn't the time budget to cover all the bases by the very thorough
> : design and testing that the additional complexity of threads would
> : require. Not to mention debugging and increased support
> : requirements.
>
> Sure, but you often program in inherrently paralizable operations at
> that level, don't you? Search a database for this, find records such
> that that, put things on the display in such-and-such format, etc, etc.
> All of which occur in processes that have threads. I mean, something
> as simple as coding a web page nowdays runs in an environment that
> has threads underlying it (or could, if it didn't), ie, a web browser.
>
> One of the things I specialize in is makefile rules. And you don't
> worry about threads when writing those (beyond a few rules of basic
> hygiene), yet when make runs, it sure finds lots of things to do at once
> on any task beyond the trivial. I certainly wouldn't want to explicitly
> code the threads for that, yeeeeesh.
Well, a slight confusion is that we're using the term _threads_ loosely;
really, we're talking general parallelization of operations on a
computer, not necessarily parallelization within a particular
application. The former doesn't really require threading at all, and
threading has a lot of drawbacks (such as the opportunity for
programmers to screw up since they're quite difficult to use strictly
properly). However, even the slouchiest of operating systems these days
has preemptive multitasking processes, and can usually exploit multiple
processors or processor cores just fine without any extra work. (I
think it's disabled in the home editions of some versions of Windows,
but that's just to force you to pay more, not because it's not there.)
That's why you can rely on make -j!
> So, just as nobody back in the day spent the time and trouble to
> actually do the finicky code to do numerical floating-point operations
> including trig, log, etc, etc, etc, but they still wrote computer programs
> that called canned routines that did these things, today programmers don't
> really write things in terms of threads very often, but they call libraries
> and services that *do* use threads extensively.
Well, sort of, at least in the sense of the threads/processes
distinction I was talking about before. Many operating systems have
their core runtimes (kernels plus other basic services) avoid the use of
threads entirely, but certainly exploit multiple processes.
netcat
d...@gatekeeper.vic.com says...
> Howard Brazee <how...@brazee.net> wrote:
> >The brain isn't so much like a Cray with difficult parallel computing
> >trying to come up with a single answer quickly. It's closer to the
> >Internet, with lots of inputs yelling at each other, and whichever
> >parts yelling loudest winning.
>
> And every so often one of the yells from the inputs resonates temporarily
> with part or all of the whole rest of the structure, and the whole thing
> rings like a bell. Meanwhile, parts of it become self-contained whirlpools
> of mutated meaning that are incomprehensible to the rest...
Sounds like mine, alright.
rgds,
netcat
Mike Ash
netcat <net...@devnull.eridani.eol.ee> wrote:
"Would you guys please SHUT UP in there!?"
PV
>What we need to do next is to run processors at faster clock-speeds,
Says who? *
--
* PV Something like badgers, something like lizards, and something
like corkscrews.
PV
>Let's assume for a moment that from now on, clock speed never increases,
>and all those transistors go into doubling the number of cores and
>memory and such every year or two. So in ten years, a typical computer
>might have a few thousand cores. Do you think a computer then will
>be doing no more computation per unit time than a computer does now?
>I suspect it'll be doing *lots* more, and a lot of it will be in terms
>of autonomous agents acting on various long-term goals.
Or a lot of "conditional execution" stuff, which is one of the uses
multiple execution pipelines get used for now. Execute both forks of an if
statement ahead of knowing the value of the expression, and then throw away
the part that becomes irrelevant.
>So. Anyways. I think throwing cores at people will, even in the
>short term and without AI or radical reorganization of software,
>make people almost as happy as throwing hertz at them used to do.
Already happening. Since no computer these days is doing only one thing, it
works better with multiple processors even if each program on it is
single-threaded. Minicomputers and up have been built that way for half a
century, and it ain't broke. *
PV
>I note that in "across realtime", one of the computing devices mentioned
>wrt Tunc Blumenthal's equipment, involved compting cores no more powerful
>than an early intel x86 chip (though, I think, faster clock rate).
>Except they were placed something one per square angstrom across the
>entire surface of Tunc's craft.
Yeah, they were used as sensors for his bobble equipment. We're getting
there! *
PV
>If you look at AI savvy-layman-level wisdom, or what the AI
>researchers tell us, they've had great expectations for... 40-50 years
>now?
You don't need AI for a singularity. Intelligence amplification works just
as well or better. *
PV
>One of the things I specialize in is makefile rules. And you don't
>worry about threads when writing those (beyond a few rules of basic
>hygiene), yet when make runs, it sure finds lots of things to do at once
>on any task beyond the trivial. I certainly wouldn't want to explicitly
>code the threads for that, yeeeeesh.
A really good example. Makes these days will chew up everything they can
get, and run frightengly faster when you add cores or processors. *
Bernard Peek
<pv+u...@pobox.com> writes
>Bernard Peek <b...@shrdlu.com> writes:
>>What we need to do next is to run processors at faster clock-speeds,
>
>Says who? *
Mostly AMD, who are currently behind the curve. Of course increasing
clock speeds is only one of the requirements.
--
Bernard Peek
Scott Lurndal
What makes you think AMD is interested in increasing clock speeds?
Istanbul has 6 cores, mangy-cours has 12.
Heat is the enemy and faster clocks lead directly to more heat dissipation.
Sure innovative cooling techniques can be developed, but ask any datacenter;
power and cooling are the two factors that any data center will attempt to
reduce.
Zoltan Somogyi
>It's interesting how people are always talking about how difficult
>multi-threaded programming is (and justifiably so) but completely gloss
>over all the difficulties involved in making new and better chips.
For most people, the difficulty of making new and better chips is simply not
relevant to them. The number of people at Intel, AMD and a few other companies
who are designing new microprocessors is counted in the hundreds, but
the number of programmers working on programs that could benefit from
parallelism (even if they currently do not) is in the hundreds of thousands,
if not millions.
Zoltan Somogyi <z...@cs.mu.OZ.AU> http://www.cs.mu.oz.au/~zs/
Department of Computer Science and Software Engineering, Univ. of Melbourne
Tim Little
> For most people, the difficulty of making new and better chips is
> simply not relevant to them. The number of people at Intel, AMD and
> a few other companies who are designing new microprocessors is
> counted in the hundreds, but the number of programmers working on
> programs that could benefit from parallelism (even if they currently
> do not) is in the hundreds of thousands, if not millions.
Yes, that's a decent argument for trying to keep processors as fast on
a single core as humanly possible for as long as possible. A 50%
direct speed boost may cost a few hundred people a few tens of
thousands of hours of design effort each. Total cost: 10 million
person-hours.
Designing all programs to use a second CPU core to achieve the same
average 50% speed boost (not all problems parallelize well) could
easily cost a few hundred thousand programmers a few hundred hours
each in extra design and debugging time. Total cost: 100 million
person-hours.
Of course by this point second cores are pretty close to "free", and
they're not going to go away. Quad-core CPUs are becoming common in
end-user systems, massively parallel video processing has been common
for years, and there are massively multicore general-purpose CPUs on
the way.
- Tim
Eivind
> Brains ARE much faster than computers. Glance at a scene, tell me what's
> in it. You can do this way faster than any computer.
Computers and brains are very very DIFFERENT. Your example above does
not establish that brains are faster, it just establishes that there
exists jobs which the brain can perform faster. But there exists plenty
of jobs that a computer can do faster too, so this line of reasoning
brings you nowhere.
Glance at an array of a million integers, output the 13th smallest. Not
even a one-second-job for a modern computer, whereas your brain will
spend days performing the same job.
Eivind
Mike Ash
Eivind <eivin...@gmail.com> wrote:
I'd be interested in any computer program which can even get this job
right, let alone fast, given that it includes "glance at".
The basics of the brain are understood well enough that the "faster than
computers" statement can be made. Each neuron is VERY roughly equivalent
to a 1MFLOPS CPU. There are 50-100 billion neurons. Multiply the two,
and there you go.
Of course the equivalence is extremely rough. But the difference in
power is so enormous right now that we can fairly safely say that the
reason a computer can perform things like multiplication so much faster
than a brain is because of the software.
David Mitchell
> In article <7kqa1iF...@mid.individual.net>,
> Eivind <eivin...@gmail.com> wrote:
>
>> Mike Ash skreiv:
>>
>> > Brains ARE much faster than computers. Glance at a scene, tell me
>> > what's in it. You can do this way faster than any computer.
>>
>> Computers and brains are very very DIFFERENT. Your example above does
>> not establish that brains are faster, it just establishes that there
>> exists jobs which the brain can perform faster. But there exists plenty
>> of jobs that a computer can do faster too, so this line of reasoning
>> brings you nowhere.
>>
>> Glance at an array of a million integers, output the 13th smallest. Not
>> even a one-second-job for a modern computer, whereas your brain will
>> spend days performing the same job.
>
> I'd be interested in any computer program which can even get this job
> right, let alone fast, given that it includes "glance at".
I think that's just a non-obvious choice of words, but I think you
get the point the OP is making.
Also, you might be interested in this:
http://www.scientificamerican.com/article.cfm?id=visionary-research&sc=rss
I'm not sure, but I think it's a story about a _much_ faster vision system.
--
=======================================================================
= David --- If you use Microsoft products, you will, inevitably, get
= Mitchell --- viruses, so please don't add me to your address book.
=======================================================================
trag
> Mike Ash <m...@mikeash.com> writes:
> >It's interesting how people are always talking about how difficult
> >multi-threaded programming is (and justifiably so) but completely gloss
> >over all the difficulties involved in making new and better chips.
>
> For most people, the difficulty of making new and better chips is simply not
> relevant to them. The number of people at Intel, AMD and a few other companies
> who are designing new microprocessors is counted in the hundreds, but
> the number of programmers working on programs that could benefit from
> parallelism (even if they currently do not) is in the hundreds of thousands,
> if not millions.
You under estimate the economic cost of making new and better chips.
Those few thousand designers are just the end of a very long
engineering chain to get better chips. Newer and better chips with
higher transistor density come on new processes with smaller feature
size. I don't have any idea how many folks it takes to create the
technology and process for a new feature size process but it may be in
the millions. The whole supply chain and machinery serving it
retools for a new process size.
Then a few thousand designers come along and design to create silicon
products on that new process.
David Johnston
>In article <rhg2e59l9goaalg46...@4ax.com>,
> David Johnston <da...@block.net> wrote:
>
>> On Fri, 23 Oct 2009 01:28:43 -0400, ZnU <z...@fake.invalid> wrote:
>>
>> >In article <JkiOdLAD...@mcdowella.demon.co.uk>,
>> > "A.G.McDowell" <mcdo...@nospam.co.uk> wrote:
>
>> >> or might be bound by limits in theoretical computing (especially if P
>> >> != NP, as many experts believe). Also, many intellectual tools whose
>> >> discovery required genius - statistics and experimental design,
>> >> axiomatic approaches, appeals to symmetry - are now routinely taught
>> >> to mortals below genius grade, so (in relative terms) de-skilling
>> >> scientific discovery. Silicon or carbon based super-geniuses will
>> >> certainly extend the region lit by light of our intelligence, but
>> >> until that happens, I don't think we will know whether what is
>> >> revealed is a garden or a desert.
>> >
>> >It's worth reflecting that there are a large number of problems on
>> >which even merely human level intelligences could probably make vastly
>> >more progress than human civilization currently does... because
>> >civilization presently allocates relatively few resources to these
>> >projects due to the high cost of humans. A PhD-level researcher, with
>> >overhead, benefits, costs... what? $150K/year? Probably more.
>> >
>> >What if a machine with that intellectual capability were a $1000
>> >off-the-shelf item, and you could rack them up in data centers?
>>
>> I kind of suspect they'd be more concerned with their problems than
>> ours.
>
>I was assuming that they'd be engineered not to be; that they'd be
>intelligent, but without much in the way of personality or outside
>interests.
>
>But even if they weren't designed this way (maybe you just can't design
>human-level intelligences like that, or maybe we'd decide it would be
>immoral), and they were granted legal equality (so you couldn't just use
>them as slave labor), they'd probably still be a lot cheaper to hire
>than human PhDs presently are, as a simple matter of supply and demand.
Hire? They are going to be the ones with all the money.
ZnU
David Johnston <da...@block.net> wrote:
Eventually. Hopefully they won't be libertarians, and we'll be able to
convince them to support a strong welfare system for us disadvantaged
meat people.
--
"The game of professional investment is intolerably boring and over-exacting to
anyone who is entirely exempt from the gambling instinct; whilst he who has it
must pay to this propensity the appropriate toll." -- John Maynard Keynes