On May 25, 9:31 am, Phil Jordan <li...@philjordan.eu> wrote:

Let's see if I can do you some justice...

Yup.  So the deadlock happened.  Ouch.

Yup, but You could make a difference.  HotSpot probably has the
highest level of 'asserts' per line of code (and a pretty darned high
level of comments per line of code) of anything Out There.  Docs are
cheaper than debugging.  But it's an aside.... back to deadlocks-in-
practice:

Crash dump?  Core file?  'ctrl-backslash' to a JVM to get a thread-
dump?
This stuff is commonly available.
If you don't have a proper tool chain, then Job One for you should be
to go get one - and I'm very serious here.  Drop everything else until
you get a functional 'path' (protocol, implementation, business
process, etc) that allows you do debug a crash from a customer in the
field.  You might need a willing beta-partner for this, hand him a
broken program & let it crash, figure out he needs disk-space to
capture the core & logs, needs training to hit 'ctrl-backslash' when
suspecting a deadlock, etc - plus FTP the files back to Home Base,
plus you need to be able to capture the files per-customer-per-bug
(ala Bugzilla), then decrypt the file (gdb for core, eyeballs or a
little perl for stack-trace logs), etc, etc, etc, ....  No Rocket
Science, but annoying bits of Engineering along with some Customer
Management.

Err.... No.

I'm not shooting for perfection here; I'm shooting for "good enough to
ship".  Which means that if - In Practice - each possible deadlock
happens once, then gets fixed - that's almost always Good Enough.
Maybe not to run a nuclear reactor, but definitely good enough to run
large businesses.  In practice, most possible deadlocks never happen -
but a few that do require several go-'rounds before getting fixed
properly.  And then the deadlock is fixed, and remains fixed.

Yup... and those folks are generally stuck anyways.  But if they are
thinking about the problem ahead of time they are going to be way way
ahead in the long run.

Sigh - we mentally missed here.

Trivial examples are.... trivial.  They can be fixed in a thousand
obvious ways.  We need to extrapolate to the non-trivial, because
that's the only place where the STM-vs-Locks argument becomes
interesting.  So lets pretend that instead of a single 'Ref _money'
and two classes, I've got 500 Ref's and a million lines of code - ala
HotSpot (Sun's JVM).  Easily >500 shared concurrent variables, about
750KLOC.  About ~100 unique locks (some are classes of striped locks)
guarding them in very complex locking patterns.  Now replace all those
locks with an STM & 'atomic'.  Is my program any more correct?  Not
really....

...I might have avoided some potential deadlocks (HotSpot uses lock
ranking asserts to avoid deadlock; deadlock rarely happens at the
engineers desk and maybe once/year in the field across all HS users).
The set of deadlocks-in-the-field avoided was miniscule.  I'll concede
that HotSpot is a rarely-well-engineered piece of concurrent code, and
that deadlocks-in-the-field appear to happen more often to other large
programs.  But still, fixing after the fact is reasonable when the
deadlock rate is so low and each fix 'sticks'.

Instead of deadlock, HS crashes far far more often because the locks
don't cover the right set of changes.  Switching out the locks for an
STM didn't change what I was guarding; it only removed any fine-
grained-lock errors (admittedly useful... but only incrementally more
so than avoiding deadlocks).  I'm still stuck with a program that's
too Big to see where the proper atomic/STM/locking boundaries need to
be.  In a trivial example I can say "go up one call level and 'atomic'
there", but in the Real Program - I can't do that.  Go up how many
layers and add 'atomic'?  1 layer?  10 layers?  100 layers?  Yes, I
see unique call-stacks with >100 layers.  I can't put 'atomic' around
'main' because that makes my program single-threaded.

Here's where I want some kind of compiler support.  'Ref' helps -
because it at least demands I have an 'atomic' around the 'Ref'.  But
'Ref' is sufficient, because a syntactally correct program simply
wraps an 'atomic' around each Ref - exact what my trivial example
did.  I'd like to be able to specify classes & groupings of Refs that
have to be wrapped in 'atomic' - that the compiler will complain
about.  Yes I'm still responsible for the semantics - I have to tell
the compiler which groupings of Refs are interesting - but I'd like
some kind of automatic support, so that as my program goes from 10
lines to 10MLOC I can be told "you touched both Ref
Person._checking._money and Ref Person._savings._money without
wrapping both Ref accesses in a single atomic, thats a datarace
error".

Cliff