race conditions

[Dave Clements]

Hey, could anyone educate me on, or does anyone have any tips
regarding race conditions?

My understanding so is that a race condition occurs when you have two
(or more) asychronous things happening, and one of these processes
relies on something happening in the other first, and because they are
async it may or may not happen, in which case whether or not it works
is conditional upon which one "wins the race".

Is there a fool proof way to debug these, or is it just when you're
noticing your code is temperamental, sometimes working sometimes not?

Are they always a bad thing? What I mean is, can you rather than
fixing the race condition, can you mitigate against it, e.g. if the
condition isn't met, roll on anyway and fix it later on down the line.
Is that a bad idea, due to the whole DRY thing?

many thanks for your kind and liberally disbursed wisdom,

dave

[Marak Squires]

I wish I could give you a good answer in only a few words, I don't think I can. :-(

The best advice I can think of would be to avoid any situation where you can find yourself in a place where don't know exactly what is going to happen. Instead of trying to debug race conditions, you should hopefully be aware enough of your application that you could identify any potential race conditions before they happen. 

You may also find an async library like https://github.com/caolan/async useful.

:-\

- Marak

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[Matt]

Also bear in mind a race condition is much more likely to happen if you run multi-process, where you can't assume you have any control over the ordering of things.

[arunoda.s...@gmail.com]

I agree with marak and best way is avoid any race condition occurances.
Use ur stuff to execute sequencely.
Use some async library or use nested callbacks.

>> nodejs+un...@googlegroups.com <nodejs%2Bunsu...@googlegroups.com>

>> For more options, visit this group at

>> http://groups.google.com/group/nodejs?hl=en?hl=en <http://groups.google.com/group/nodejs?hl=en?hl=en>

>
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to

> nodejs+un...@googlegroups.com <nodejs%2Bunsu...@googlegroups.com>

> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>

--
Arunoda Susiripala

@arunoda

https://github.com/arunoda

http://www.linkedin.com/in/arunoda

[Ryan Schmidt]

On Sep 26, 2011, at 22:33, Dave Clements wrote:

> Hey, could anyone educate me on, or does anyone have any tips
> regarding race conditions?
>
> My understanding so is that a race condition occurs when you have two
> (or more) asychronous things happening, and one of these processes
> relies on something happening in the other first, and because they are
> async it may or may not happen,

...in the expected / needed order...

> in which case whether or not it works
> is conditional upon which one "wins the race".
>
> Is there a fool proof way to debug these, or is it just when you're
> noticing your code is temperamental, sometimes working sometimes not?
>
> Are they always a bad thing? What I mean is, can you rather than
> fixing the race condition, can you mitigate against it, e.g. if the
> condition isn't met, roll on anyway and fix it later on down the line.
> Is that a bad idea, due to the whole DRY thing?

While not an authoritative source, Wikipedia defines a race condition as "a flaw in [a] process whereby the output and/or result of the process is unexpectedly and critically dependent on the sequence or timing of other events."

http://en.wikipedia.org/wiki/Race_condition#Computing

So yes, I would say flaws are always a bad thing.

Once you become aware of the race condition in your code, you can correct it. We don't know what your code looks like so we can't propose methods to fix it. But either your code has a race condition (the order in which events fire could cause things in your app to go wrong), or it does not have a race condition (any order of events will result in a successful outcome); there's no in-between.

[Ben Blair]

Just to emphasize what Ryan says, avoiding race conditions means
"**any** order of events will result in a successful outcome"
(emphasis added). If you're just getting started with asynchronous
programming, it is sometimes tempting to try to avoid race conditions
by forcing events to happen in a particular order. In my experience,
that introduces more bugs than it avoids. Much better to make as few
assumptions as possible about in what order events will occur. Your
code will end up much simpler.

- Ben

> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com

[Robert]

Race hazards arise whenever two or more activities interact whereby
some interactions produce desired results and others produce unwanted
results. Sometimes you win, sometimes you loose. BTW if the result is
always good, rather than being a race hazard, the race condition would
be "don't care", also known as fully parallel and as fully
independent. Race hazards required some sort of shared dependency.

By definition, a race hazard is always risky and the nature what is
interacting determines the serverity of the consequences. Thus race
hazards are to be avoid in good design by carefully managing the
underlying dependencies, especially software system design where a
critical single bit error can bring down an entire system.

In software systems typically the activities (software "processes" in
the general sense) are contending over shared resources and disturb
each's intermediate results. The most common symptom is a working
system that sometimes mysteriously misbehaves. Typically this is
difficult to debug from scratch because a "repro" may be very hard
thanks to the variable nature of the race hazard.

The problem is not just limited to the order of interaction
(serialization), it can also involve the disruption of incomplete
results most often solved by some sort of locking/queueing mechanism.
This itself forces a type of dynamically imposed serialization where a
race hazard would otherwise be in play. A real life Node JS example is
being discussed in the "in order of processing events with asysnc
behavior" thread in this group . In fact by returning to the behavior
most closely resembling queued hardware interrupts, Node recreates all
these issues at a higher level of abstraction!

Golden oldie: if you are into seminal works, the still readable grand
daddy of them all is Djistra's Co-operating Sequential Processes which
still does an excellent job of setting the problem, although the field
has come a long way since then in solving it!

On Sep 26, 8:33 pm, Dave Clements <huperekch...@googlemail.com> wrote:
> Hey, could anyone educate me on, or does anyone have any tips
> regarding race conditions?
>
> My understanding so is that a race condition occurs when you have two
> (or more) asychronous things happening, and one of these processes

... snip for space

[Dominic Tarr]

since node's callback style makes async ctrlflow very explicit, race conditions are really not the problem in node that they are in multi-threaded languages.

if you did had something like this:

createDatabase(dbname, function (){}) //just ignored the callback

saveItem(dbname, obj, function () {})

that *could* work if createDatabase is ready before saveItem gets to the db bit.

if you are coding properly, and so are your dependencies:

createDatabase(dbname, function (){

  saveItem(dbname, obj, function () {})

}) 

you will not be vulnerable to race conditions.

[Bruno Jouhier]

What about this:

var lastId = 0;

exports.saveObj = function(obj, cb) {
lastId++;
openDatabase(dbname, function (db){
obj.id = lastId; // you LOSE!
db.saveItem(obj, cb);
})

You have a race condition because lastId is incremented from one
event, and read from another event.

The fix is obvious (obj.id = ++lastId) and node makes it very easy to
reason about race conditions because of its single threaded nature.
But race conditions do exist and you have to be careful as soon as you
share state across callbacks.

Regarding the OP, you have two ways to approach programming: a
rigorous one in which you know precisely what your code is supposed to
do, what is shared and what is not shared, what is immutable and what
is not, you care about invariants, etc. and a lousy one in which you
just write stuff, fix it until it works and that's it. If you want to
survive in an async world (or a threaded world, as well), you have no
choice, you have to be rigorous about every line of code you write
(the lastId++ line above look innocuous but it is not!). So, don't try
to mitigate (race conditions may happen but I'll deal with them), be
rigorous in the first place.

Bruno

[Daniel Ly]

On Wednesday, 28 September 2011 08:46:20 UTC+2, Bruno Jouhier wrote:

var lastId = 0;

exports.saveObj = function(obj, cb) {
  lastId++;
  openDatabase(dbname, function (db){
    obj.id = lastId; // you LOSE!
    db.saveItem(obj, cb);
})

You have a race condition because lastId is incremented from one
event, and read from another event.

I don't see a race condition here. lastId++ is always incremented before being saved to obj.id.

Maybe this a bit contrived example?

var accessCounter = 0; 

exports.saveObj = function(obj, cb) { 
  openDatabase(dbname, function (db){ 
    obj.counter = ++accessCounter; 
    db.saveItem(obj, cb); 
})

exports.logAccessCounter = function() { 
  accessCounter++;

  

  console.log("accessCounter", accessCounter);
}

and somewhere else:

saveObj(obj, db);

logAccessCounter();

what is printed in the log or saved in the database depends on the race.

[Bruno Jouhier]

On Sep 29, 8:39 am, Daniel Ly <nal...@gmail.com> wrote:
> On Wednesday, 28 September 2011 08:46:20 UTC+2, Bruno Jouhier wrote:
>
> > var lastId = 0;
>
> > exports.saveObj = function(obj, cb) {
> >   lastId++;
> >   openDatabase(dbname, function (db){
> >     obj.id = lastId; // you LOSE!
> >     db.saveItem(obj, cb);
> > })
>
> > You have a race condition because lastId is incremented from one
> > event, and read from another event.
>
> I don't see a race condition here. lastId++ is always incremented before
> being saved to obj.id.

There IS a race condition!

Let us suppose that you have 2 HTTP requests R1 and R2 that come in
and that they both try to save a new object to the db with this
implementation.

If the requests are far apart, you will have the following sequence of
events:

R1/lastId++
R1/obj.id = lastId; db.saveItem(obj, cb)
R2/lastId++
R2/obj.id = lastId; db.saveItem(obj, cb)

Fine: the objects will get different ids.

If the requests are close enough, the sequence could be:

R1/lastId++
R2/lastId++
R1/obj.id = lastId; db.saveItem(obj, cb)
R2/obj.id = lastId; db.saveItem(obj, cb)

The two objects will get the same id, which is probably not what you
want. You'll get an integrity constraint violation if id has a UNIQUE
constraint in the db.

So, don't assume that you are safe because you use nested callbacks.
This is true if you only manipulate state maintained in your closures,
but not if you start to access shared state (lastId in the example)

Bruno

[Naouak]

In the lastid example you can simply avoid that behavior by passing a copy of lastid to the callback. Simple trick that you may already use for loop.

[Daniel Ly]

I wrote:

 

> I don't see a race condition here. lastId++ is always incremented before
> being saved to obj.id.

Bruno wrote:

 

There IS a race condition!

Let us suppose that you have 2 HTTP requests R1 and R2 that come in
and that they both try to save a new object to the db with this
implementation.

Ah, of course... Now I understand... 2 HTTP requests... of course...

[Dave Clements]

On Sep 29, 3:36 am, Naouak <tard...@gmail.com> wrote:
> In the lastid example you can simply avoid that behavior by passing a copy
> of lastid to the callback. Simple trick that you may already use for loop.

this idea is sort of what I meant by mitigating, although I would
admit that a laziness came into as well, as in "why work it all out
when I can just put a plaster on it" temptation. But, that would
probably just create a problem elsewhere and as a coding philosophy is
prone to issues.

these examples are great though, especially in how to deal with them,
I can't remember off the top of my head the exact code I was looking
at and wondering "is this creating a race condition", but I'll see if
I can dig it out, if I do locate it, I'll be sure to add it to the
thread.

thanks guys,

Dave