Re: Problem with XP scheduler?
J de Boyne Pollard
MS> component of Windows NT 3.51, such that if the
MS> CSRSS.EXE process faults the entire Windows NT
MS> operating system is shutdown."
PB> The phrase "is considered" is particularly interesting. It is
PB> as if whoever wrote it knows it is more of a considered
PB> opinion than a fact.
It is, nonetheless, a fact, not an opinion. From Windows NT 5
onwards, the BreakOnTermination flag is set in the EPROCESS structure
for that process. In earlier versions, the Session Manager halts the
system when it detects that a CSRSS process has died.
J de Boyne Pollard
DB> multitasking means different things to different people.
DB> If you are an old timer like me, there were major
DB> screams 30 years ago when Unix claimed to be
DB> multitasking. At that time most companies described
DB> what we now consider as multitasking as
DB> multiprocessing and that tasks were what we currently
DB> call threads. And 30 years ago, Unix did not support
DB> threads.
Yes (although with either definition the trolling here is false).
More fundamentally, "task" means different things to different
people. In some cases, a "task" is what in the Windows NT model would
be a "thread". In other cases, a "task" would be a Windows NT
"process". In yet other cases still, a "task" is what in the Solaris
model would be a "light weight process". Teminology is and long has
been, most definitely, an issue in this area. If one has clear
definitions of "thread", "process", and "processor", then "multi-
thread", "multi-process", and "multi-processor" have clear meanings.
But since "task" is often ambiguous, "multi-tasking" is as well.
Let's not get started on the difference between "concurrent" and
"multi-programmed" or what "job" means ... (-:
Anne & Lynn Wheeler
J de Boyne Pollard <J.deBoyn...@Tesco.NET> writes:
> Yes (although with either definition the trolling here is false).
> More fundamentally, "task" means different things to different
> people. In some cases, a "task" is what in the Windows NT model would
> be a "thread". In other cases, a "task" would be a Windows NT
> "process". In yet other cases still, a "task" is what in the Solaris
> model would be a "light weight process". Teminology is and long has
> been, most definitely, an issue in this area. If one has clear
> definitions of "thread", "process", and "processor", then "multi-
> thread", "multi-process", and "multi-processor" have clear meanings.
> But since "task" is often ambiguous, "multi-tasking" is as well.
> Let's not get started on the difference between "concurrent" and
> "multi-programmed" or what "job" means ... (-:
multiprogramming was mainframe ... more lightweight "threads" within the
same address space. multi-tasking could be more heavy weight
... and could involve different address spaces. multiprocessing
tended to be multiple real processors.
there was "tightly-coupled" (shared-memory) multiple processors
... there was also "loosely-coupled" (non-shared-memory) but possibly
shared disk or other i/o, multiple processors (clusters) ... and then
there are the "shared-nothing" clusters.
in the 90s ... sequent claimed that it did much of the "windows" scaleup
work for shared-memory multiple processors (getting NT running
efficiently on more than two-processor multiple processor machines.
when charlie was working on fine-grain multiprocessor locking for cp67
kernel (late 60s), he invented compare&swap instruction (name of
instruction chosen because CAS are his initials). The attempt to get it
included in 370 (mainframe) multiprocessor was initially rebuffed
... claiming that TEST&SET instruction was more than sufficient. The
challenge was to come up with uses for compare&swap instruction that
weren't multiprocessor specific. Thus was born the examples of using
compare&swap in multithreaded/multiprogrammed applications ... to
coordinate the different threads ... regardless of whether the
underlying hardware was just a single processor or multiple processors.
It became standard use for highly optimized, multithreaded subsystems
... like various kinds of database management systems (DBMS) ... and
started showing up on processors other than 370s.
description from current mainframe principles of operation (much of it
the original justification used for compare&swap in early 70s):
A.6 Multiprogramming and Multiprocessing Examples
http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/dz9zr003/A.6?DT=20040504121320
scheduling can be somewhat orthogonal ... I had done dynamic adaptive
resource management for cp67 as undergraduate in the 60s (sometimes
referred to as the "fair share scheduler" because default resource
management policy was "fair share".
--
40+yrs virtualization experience (since Jan68), online at home since Mar1970
Anne & Lynn Wheeler
re:
http://www.garlic.com/~lynn/2009s.html#54 Problem with XP scheduler?
actually, in the mid to late 80s (>20 years ago) looking at some of the
unix scheduler code ... I realized that I had rewritten it nearly two
decades previously (over 40 yrs ago) in cp67.
I attribute it to possibly have originated in CTSS. Some of the CTSS
people went to science center on the 4th flr ... and did (virtual
machine) cp40 ... which then morphed into (virtual machine) cp67. Other
of the CTSS people went to Multics on the 5th flr ... and there is
various folklore regarding Multics and UNIX.
In any case, when some people from the science center came out and
installed cp67 at the univ in Jan68 ... I got to do lots of
changes/enhancements. One was rewritting the scheduling infrastructure
... part of that original cp67 scheduling implementation (that I
rewrote) bore some amount of similarity to the unix scheduling
implementation that I later ran across in the mid & late 80s.
misc. past posts mentioning fair share scheduler
http://www.garlic.com/~lynn/subtopic.html#fairshare
that have been comments in the past about multi-core being a solution
for poorly implemented schedulers ... as long as there are at least as
many different core/processors as there are things to run ... then a
scheduler is never faced with decision about what *NOT* to run (at any
particular moment) ... sort of the inverse of scheduling decision about
what to run ... as long as there are enough processors to run
everything.
tanix
>DB> My real complaint with this whole discussion is that
>DB> If you are an old timer like me, there were major
>DB> screams 30 years ago when Unix claimed to be
>DB> multitasking. At that time most companies described
>DB> what we now consider as multitasking as
>DB> multiprocessing and that tasks were what we currently
>DB> threads.
>
>Yes (although with either definition the trolling here is false).
>More fundamentally, "task" means different things to different
>people. In some cases, a "task" is what in the Windows NT model would
>be a "thread". In other cases, a "task" would be a Windows NT
>"process". In yet other cases still, a "task" is what in the Solaris
>model would be a "light weight process". Teminology is and long has
>been, most definitely, an issue in this area. If one has clear
>definitions of "thread", "process", and "processor", then "multi-
>thread", "multi-process", and "multi-processor" have clear meanings.
>But since "task" is often ambiguous, "multi-tasking" is as well.
>Let's not get started on the difference between "concurrent" and
>"multi-programmed" or what "job" means ... (-:
Well, that is all fine and dandy.
But...
What it boils down to, no matter what you call it,
is something that can have potentially deadly effect on operation
of your entire system, and I see it all day long every day with
windows.
When I have at least 10 programs running and about 30 windows
opened and have done several hours worth of work and click back
on my firefox icon and do a refresh of about 10 pages in a rapid
succession, and my box simply freezes, what it means to me is
that I lost ALL sorts of work, and I lost ALL sorts of program
states, and it is going to take me half an hour to restore all
things, and that is if I am lucky. Some work might have been
lost for good.
I could care less if firefox freezes by itself but I can continue
with everything else. I could simply hand kill that process from
the task manager need be as I have done gazillion of times.
But when I loose nothing less then OS, than it is a TOTALLY
different issue, and the issue is OS, and not some screwed up
firefox or some driver it happens to be using.
Simple as that.
--
Programmer's Goldmine collections:
Tens of thousands of code examples and expert discussions on
C++, MFC, VC, ATL, STL, templates, Java, Python, Javascript,
organized by major topics of language, tools, methods, techniques.
Maxim S. Shatskih
> work for shared-memory multiple processors (getting NT running
> efficiently on more than two-processor multiple processor machines.
Can you point at particular legacy of this Sequent's effort in Windows?
For me, looks like that the "so-so more-or-less" SMP support was in Windows from the beginning, and the major improvements in it were done only in 2008/2008 R2 timeframe.
--
Maxim S. Shatskih
Windows DDK MVP
ma...@storagecraft.com
http://www.storagecraft.com
Maxim S. Shatskih
> on my firefox icon and do a refresh of about 10 pages in a rapid
> succession, and my box simply freezes
I think this is due to Firefox playing bad games with Windows shell, which is - in such a scenario - the only component which freezes.
IE is better in this. IE crashes sometimes though - mainly in Flash.ocx.
jmfbahciv
>> in the 90s ... sequent claimed that it did much of the "windows" scaleup
>> work for shared-memory multiple processors (getting NT running
>> efficiently on more than two-processor multiple processor machines.
>
> Can you point at particular legacy of this Sequent's effort in Windows?
[inserting <CRLF>s required]
>
> For me, looks like that the "so-so more-or-less" SMP support
>was in Windows
Nope.
> from the beginning, and the major improvements in
>it were done only in 2008/2008 R2 timeframe.
>
/BAH
jmfbahciv
Does the monitor take a dump?
/BAH
Anne & Lynn Wheeler
"Maxim S. Shatskih" <ma...@storagecraft.com.no.spam> writes:
> Can you point at particular legacy of this Sequent's effort in Windows?
>
> For me, looks like that the "so-so more-or-less" SMP support was in
> Windows from the beginning, and the major improvements in it were done
> only in 2008/2008 R2 timeframe.
re:
http://www.garlic.com/~lynn/2009s.html#54 Problem with XP scheduler?
http://www.garlic.com/~lynn/2009s.html#55 Problem with XP scheduler?
Sequent had 16-way & 32-way ... and had done a lot of work on Unix to
improve Unix scaleup in more than 2-way & 4-way operation (dynix).
before we left ibm in early 90s, we had done some work with SCI ... for
possibility for ha/cmp scaleup ... although at the time we were dealing
with processor chip that didn't provide for any cache consistency
... and all scaleup had to be "cluster". old reference to jan92 meeting
in ellison's conference room on ha/cmp cluster scaleup
http://www.garlic.com/~lynn/95.html#13
other posts mentioning ha/cmp product
http://www.garlic.com/~lynn/subtopic.html#hacmp
later, two of the people (at the jan92 meeting) had left and show up at
small client/server startup responsible for something called "commerce
server" (the startup had also invented something called "SSL"). we were
brought in because they wanted to payment transactions on their server
... and the result is now frequently called "electronic commerce". Part
of the "electronic commerce" work was something called a "payment
gateway" ... which acted as interface between webservers on the internet
and financial networks for payment transactions (we periodically refer
to it as original SOA). The initial "payment gateway" was an HA/CMP
configuration with several other boxes around the edges providing
various kinds of integrity, diagnostic, and security functions.
misc. past posts mentioning "electronic commerce" and "payment gateway"
work:
http://www.garlic.com/~lynn/subtopic.html#gateway
the small client/server startup had also a growing presence on the
internet ... with servers deliverying their client & server products.
They were using (unix) servers from companies in the silicon valley area
... that were quickly overloaded and required installation of more &
more servers ... with their own unique host name ... and internet
customers were asked to selective specify different URL host names when
connecting (trying to spread out the internet load, this was before some
of the front-end router work that was done at google). then they brought
in a large sequent box and the problems went away (not just large
sequent box, and dynix smp scaleup work ... but dynix also had some
amount of tcp/ip protocol scale-up work)
At the time NT had some SMP support ... but running on 8-way didn't show
any increased thruput than running on 4-way (and little thruput
improvement on 4-way compared to 2-way). Somehow sequent was involved to
get NT running on their 32-way box ... and do a lot of scaleup work to
show increasing thruput as configurations scaled passed 4-way (aka more
than four processors) in SMP configuration.
Later in the 90s, sequent was doing a 256-way smp SCI-based machine
(NUMA-Q) and doing further work on their (dynix) unix to scaleup to
256-way (although I don't know of any work on NT for 256-way). In that
timeframe, Steve Chen (from cray & chen supercomputers) was CTO at
sequent ... and we did some consulting for him.
Now while sequent did a lot of work on NT kernel to show increasing
thruput as processors increased past two ... (at the time) NT thruput
still didn't match Unix products (on the same hardware). at one point,
there was a joint project that redmond had for putting a large web-based
service on the internet and we were brought in to do some work on it.
We showed that NT still didn't have thruput necessary to support the
fully deployed operation ... and the redmond group decided that I would
be the person to explain to their CEO why a UNIX platform would have to
be used for the deployment. Before that actually took place, the
executive running the group ... decided that (instead) the web service
would have a staged roll-out ... the web population supported would
never exceed the scaleup capability of NT (and therefor it would not be
necessary to use a UNIX platform ... and I wouldn't have to explain to
their CEO why NT wasn't being used).
now there are two somewhat related issues (but not identical)
... whether NT showed increasing (SMP) thruput as the number of
processors increased ... and NT thruput compared to other SMP
implementations on the same hardware.
sequent wiki page:
http://en.wikipedia.org/wiki/Sequent_Computer_Systems
23May95 article:
Sequent Unveils New High-End Systems for Windows NT; Based on
Industry-Leading Platform Symmetry...
http://www.allbusiness.com/technology/software-services-applications-computer/7126055-1.html
from above:
The WinServer systems add higher-end Windows NT-based performance to the
existing integrated solutions Sequent provides to support customers'
business requirements in decision support, online transaction processing
and messaging. WinServer systems offer customers the benefits of proven
Symmetry hardware, the industry's most mature and technologically
advanced SMP platform, which has been installed with the UNIX operating
system at thousands of Sequent customer sites around the world.
... snip ...
9Nov92 article:
JUST WHEN SEQUENT THOUGHT IT WAS SAFE...
http://www.businessweek.com/archives/1992/b329273.arc.htm
from above:
Microsoft Corp. has picked Sequent multiprocessing technology for
Windows NT, the advanced operating system software it is readying for
1993.
... snip ...
steve chen wiki page:
http://en.wikipedia.org/wiki/Steve_Chen_%28computer_engineer%29
note the reference to SCI in the above ... is different than
the SCI technology used by Convex, SGI, Sequent, DG & others
for scalable shared memory multiprocessors ... wiki page:
http://en.wikipedia.org/wiki/Scalable_Coherent_Interface
above makes mention that the standardization effort eventually morphed
into current InfiniBand ("which is similar to SCI in many ways")
infiniband wiki:
http://en.wikipedia.org/wiki/InfiniBand
for some topic drift ... only marginally related posts (security,
not SMP):
http://www.garlic.com/~lynn/2009l.html#20 Cyber attackers empty business accounts in minutes
other earlier posts mentioning the above:
http://www.garlic.com/~lynn/2009.html#60 The 25 Most Dangerous Programming Errors
http://www.garlic.com/~lynn/2009g.html#18 Top 10 Cybersecurity Threats for 2009, will they cause creation of highly-secure Corporate-wide Intranets?
http://www.garlic.com/~lynn/2009h.html#28 Computer virus strikes US Marshals, FBI affected
http://www.garlic.com/~lynn/2009i.html#22 My Vintage Dream PC
Anne & Lynn Wheeler
re:
http://www.garlic.com/~lynn/2009s.html#54 Problem with XP scheduler?
http://www.garlic.com/~lynn/2009s.html#55 Problem with XP scheduler?
the folklore in a.f.c. is that NT starts out as VMS by some people hired
from DEC. VMS had specialized in some amount of commercial
dataprocessing ... but didn't particularly have very long SMP support
heritage. post with old email about vax/vms SMP product announcements
http://www.garlic.com/~lynn/2007.html#email880324
http://www.garlic.com/~lynn/2007.html#email880329
in this post
http://www.garlic.com/~lynn/2007.html#46 How many 36-bit Unix ports in the old days?
then windows continues as desktop platform and NT becomes the (somewhat
compatible) server platform. then when the two platforms were
consolidated ... more was taken from the desktop platform than the
server platform ... possibly implying that some amount of the SMP work
was dropped(?).
this possibly accounts for the stories in the press about intel having
to explain to the CEO in redmond why single processor chips couldn't
just continue to get faster ... and why there was the move to multi-core
(multiprocessor) chips ... AND why windows (& desktop applications)
would have to significantly improve its SMP support ... referenced in
this article:
http://www.theregister.co.uk/2007/05/01/mundie_mundie/
mentioned in this post
http://www.garlic.com/~lynn/2007i.html#78 John W. Backus, 82, Fortran developer, dies
note ... a.f.c. tends to have some amount of topic drift ... when I was
at SJR, Backus' office was just a couple doors down the corridor. In
any case, above post has references to old email regarding boca/os2
group asking me about "scheduling":
http://www.garlic.com/~lynn/2007i.html#email871204
http://www.garlic.com/~lynn/2007i.html#email871204b
in this post
http://www.garlic.com/~lynn/2007i.html#60 John W. Backus, 82, Fortran developer, dies
In any case, some of the decktop platform smp/multicore was that a lot
of the desktop apps were strictly single-threaded and only ran faster
when the processor got faster (additional processors didn't help,
multiple processors help multiple different applications to run
concurrently, but didn't help with running a single-threaded application
faster) ... and for these apps to show increased thruput ... they would
have to be rewritten for multi-thread (&/or parallel) operation.
for other topic drift ... some past posts about working on design for
5-way SMP in the mid-70s (which got canceled before being announced):
http://www.garlic.com/~lynn/submain.html#bounce
which was almost immediately followed by working on design for 16-way
SMP ... which also got canceled before being announced ... some recent
references:
http://www.garlic.com/~lynn/2009o.html#10 Microprocessors with Definable MIcrocode
http://www.garlic.com/~lynn/2009o.html#14 Microprocessors with Definable MIcrocode
http://www.garlic.com/~lynn/2009o.html#17 Broken hardware was Re: Broken Brancher
http://www.garlic.com/~lynn/2009q.html#74 Now is time for banks to replace core system according to Accenture
http://www.garlic.com/~lynn/2009s.html#32 Larrabee delayed: anyone know what's happening?
and previous mention ... lots of past posts mentioning SMP (and/or
compare&swap instruction)
http://www.garlic.com/~lynn/subtopic.html#smp
Don Burn
"Anne & Lynn Wheeler" <ly...@garlic.com> wrote in message
news:m3hbr76...@garlic.com...
>
> this possibly accounts for the stories in the press about intel having
> to explain to the CEO in redmond why single processor chips couldn't
> just continue to get faster ... and why there was the move to multi-core
> (multiprocessor) chips ... AND why windows (& desktop applications)
> would have to significantly improve its SMP support ... referenced in
> this article:
> http://www.theregister.co.uk/2007/05/01/mundie_mundie/
>
It is ironic that Gelsinger complains about single processor chips, years
before it was Intel that pushed Microsoft to release that disaster known as
Windows ME because they were concerned about the lower end multi-processor
systems taking business away from the high end Pentiums. I knew people from
the Windows NT space that were discusted with the large amount of $$$ that
Intel threw at the ME group to get it out and promoted.
--
Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply
__________ Information from ESET NOD32 Antivirus, version of virus signature database 4732 (20091231) __________
The message was checked by ESET NOD32 Antivirus.
Peter Flass
> re:
> http://www.garlic.com/~lynn/2009s.html#54 Problem with XP scheduler?
> http://www.garlic.com/~lynn/2009s.html#55 Problem with XP scheduler?
> http://www.garlic.com/~lynn/2009s.html#59 Problem with XP scheduler?
>
> the folklore in a.f.c. is that NT starts out as VMS by some people hired
> from DEC. VMS had specialized in some amount of commercial
> dataprocessing ... but didn't particularly have very long SMP support
> heritage. post with old email about vax/vms SMP product announcements
> http://www.garlic.com/~lynn/2007.html#email880324
> http://www.garlic.com/~lynn/2007.html#email880329
> in this post
> http://www.garlic.com/~lynn/2007.html#46 How many 36-bit Unix ports in the old days?
According to Pascal Zachary, and some memos on Bitsavers, it appears
Cutler & Co. were working on a new OS for a new system (Prism/Mica) at
DECWRL. When DEC, or should I say Digital, canceled that Cutler was
happy to move to M$, where he could still develop a brand-new OS from
scratch.
Anne & Lynn Wheeler
Peter Flass <Peter...@Yahoo.com> writes:
> According to Pascal Zachary, and some memos on Bitsavers, it appears
> Cutler & Co. were working on a new OS for a new system (Prism/Mica) at
> DECWRL. When DEC, or should I say Digital, canceled that Cutler was
> happy to move to M$, where he could still develop a brand-new OS from
> scratch.
re:
http://www.garlic.com/~lynn/2009s.html#54 Problem with XP scheduler?
http://www.garlic.com/~lynn/2009s.html#55 Problem with XP scheduler?
http://www.garlic.com/~lynn/2009s.html#59 Problem with XP scheduler?
http://www.garlic.com/~lynn/2009s.html#62 Problem with XP scheduler?
thnks, comments I had seen were DEC (former VMS) people going to m'soft
to do NT ... and lots of DEC technology possibly being part of NT. The
main point was that people didn't have long SMP history contributing to
decision involving sequent in NT SMP scaleup. This isn't exactly the
same as straight performance (i.e. NT having equivalent thruput to
UNIXes of the period on the same hardware) ... but is likely work on
finer-grain locks in the kernel ... so adding processors showed some
incremental thruput (and weren't spending all their time
waiting/spinning on locks). Some of the (other) performance issues was
implied in the comments about weeks away from having to explain to their
CEO that a high-end UNIX would be used for some deployment (rather than
NT) ... until the responsible executive changed the definition of the
deployment to be only that which NT could do.
when charlie was doing work on cp67 fine-grain locking and invented the
compare&swap instruction ... one of the scenarios was showing
compare&swap directly updating variables ... as opposed to (barrier)
locks around the updating of the variables (scenario for various kinds
of applications like database management systems ... for multithreaded
work ... whether running in a multiprocessor environment or not). misc.
past posts mentioning smp &/or compare&swap instruction
http://www.garlic.com/~lynn/subtopic.html#smp
The recent Larrabee thread in comp.arch has touched on wide variety
of topics ... but yesterday there were couple posts discussing
compare&swap variations. misc. past posts in the Larrabee thread:
http://www.garlic.com/~lynn/2009s.html#18 Larrabee delayed: anyone know what's happening?
http://www.garlic.com/~lynn/2009s.html#20 Larrabee delayed: anyone know what's happening?
http://www.garlic.com/~lynn/2009s.html#22 Larrabee delayed: anyone know what's happening?
http://www.garlic.com/~lynn/2009s.html#23 Larrabee delayed: anyone know what's happening?
http://www.garlic.com/~lynn/2009s.html#29 channels, was Larrabee delayed: anyone know what's happening?
http://www.garlic.com/~lynn/2009s.html#30 Larrabee delayed: anyone know what's happening?
http://www.garlic.com/~lynn/2009s.html#31 Larrabee delayed: anyone know what's happening?
http://www.garlic.com/~lynn/2009s.html#32 Larrabee delayed: anyone know what's happening?
http://www.garlic.com/~lynn/2009s.html#33 Larrabee delayed: anyone know what's happening?
http://www.garlic.com/~lynn/2009s.html#34 Larrabee delayed: anyone know what's happening?
http://www.garlic.com/~lynn/2009s.html#35 Larrabee delayed: anyone know what's happening?
http://www.garlic.com/~lynn/2009s.html#36 Larrabee delayed: anyone know what's happening?
http://www.garlic.com/~lynn/2009s.html#42 Larrabee delayed: anyone know what's happening?
http://www.garlic.com/~lynn/2009s.html#43 Larrabee delayed: anyone know what's happening?
http://www.garlic.com/~lynn/2009s.html#48 Larrabee delayed: anyone know what's happening?
... couple weeks and it has been 42yrs since I first got to play with cp67