Received: by 10.236.86.2 with SMTP id v2mr2091967yhe.36.1352932120219;
Wed, 14 Nov 2012 14:28:40 -0800 (PST)
X-BeenThere: nodejs@googlegroups.com
Received: by 10.49.63.40 with SMTP id d8ls12371qes.44.gmail; Wed, 14 Nov 2012
14:28:21 -0800 (PST)
Received: by 10.49.58.167 with SMTP id s7mr12293qeq.5.1352932101806;
Wed, 14 Nov 2012 14:28:21 -0800 (PST)
Date: Wed, 14 Nov 2012 14:28:20 -0800 (PST)
From: pie_lard <pie_l...@yahoo.com>
To: nodejs@googlegroups.com
Message-Id: <bf841e2f-93a2-4363-bb93-73cfe7270931@googlegroups.com>
In-Reply-To: <8F3E2B29D1344A98A20CE3ED12FCD5D3@newrelic.com>
References: <b56d231b-a521-4090-bf24-c9d63ecc0124@googlegroups.com>
<7FA06F0F70B74ECABF01ED3B02CF4616@newrelic.com>
<ee76aa2f-1554-4b54-9ff3-f0cb84b331c5@googlegroups.com>
<15ACBB49A5A64117B57629E4AEB0004E@newrelic.com>
<176915b8-50f3-4ef1-8736-5c8338a4602f@googlegroups.com>
<5705122A59C34C70BC226BE157B5B1B8@newrelic.com>
<7d07ff61-95eb-4050-98a7-3cd0f67c18e8@googlegroups.com>
<D22D2E2533124BE18043F232566AD013@newrelic.com>
<bd2993ea-ec56-4fb1-bc58-a567a6b0df30@googlegroups.com>
<8F3E2B29D1344A98A20CE3ED12FCD5D3@newrelic.com>
Subject: Re: [nodejs] Domains and connection pools
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="----=_Part_1120_19438936.1352932101014"
------=_Part_1120_19438936.1352932101014
Content-Type: multipart/alternative;
boundary="----=_Part_1121_5565958.1352932101021"
------=_Part_1121_5565958.1352932101021
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
On Wednesday, 14 November 2012 22:04:15 UTC, Forrest L Norvell wrote:
>
> =20
> =20
> Forrest L Norvell
> Node.js agent engineer
> | E for...@newrelic.com <javascript:> | C (415) 823=E2=80=916356 | T @oth=
iym23<http://twitter.com/othiym23>
> | G github.com/othiym23 | W newrelic.com
> *(* *(* =E2=80=A2*)**)* *New Relic*
>
> On Wednesday, November 14, 2012 at 1:35 PM, pie_lard wrote:
>
> Suppose node were changed to remove the feature that lets domains track=
=20
> EventEmitters. So domain.add(), .remove() and .members would all be=20
> removed. The code that automatically adds new EventEmitters to the activ=
e=20
> domain (if there is one) would also be deleted. Finally, suppose .bind()=
=20
> and .intercept() were also removed.
>
> As an aside, you should only have to call domain.{add,remove} on a handle=
=20
> or an emitter yourself under very specific (and rare) conditions.
>
> That would just leave domain.run() - plus the ability to create new=20
> domains and possibly nest them.
>
> Would that be enough to effectively recreate synchronous stack-based=20
> exception handling?
>
> No, and I'll explain why below.=20
>
> My answer would be yes - and in fact it would remove my connection-pool=
=20
> problem altogether without any need for the 3rd-party module authors to=
=20
> change anything.
>
> To put it another way; in what real-world circumstances would you need th=
e=20
> features I just removed?
>
> Please understand I ask this as a bit of a node noob ;) It's not a=20
> criticism of domains - I would like someone to explain why I'm wrong abou=
t=20
> this.
>
> The problem is it's impossible to know in advance whether a chunk of code=
=20
> is going to use EventEmitters or not, and without domain code there to wr=
ap=20
> them, you'd lose the necessary state to tie a specific request to a=20
> specific domain. Consider the following (buggy / incomplete) Express=20
> handler:
>
> app.get('/test', function (req, res) {
> var d =3D domain.create();
> d.on('error', function (error) { res.send(500, {"content-type":"text/pl=
ain"}, error.stack); });
> d.run(function () {
> fs.readFile(FILENAME, 'utf-8', function (err, contents) {
> // process.domain should be available here
> res.send(contents.replace(/:/gm, '/'));
> });
> });
> });
>
>
> The call to res.send is going to happen several passes through the event=
=20
> loop after the initial handler is called. It's entirely possible that in=
=20
> the time the file is read from disk, another several requests to that=20
> handler are going to have come in to the server. Without dealing with the=
=20
> EventEmitters, how will any errors emitted inside the fs.readFile callbac=
k=20
> be connected back to the enclosing domain?
>
> Maybe this is what I'm misunderstanding! I was under the impression that=
=20
your example would in fact work (ie. that your domain handler would catch=
=20
an error thrown by the res.send() call).
The docs say:
Additionally, callbacks passed to lowlevel event loop requests (such as to=
=20
fs.open, or other callback-taking methods) will automatically be bound to=
=20
the active domain. If they throw, then the domain will catch the error.
Doesn't that imply that you've done all you need to do in the above code?=
=20
That just by starting the chain of callbacks inside run() that the active=
=20
domain will persist through them? And that as you're creating a new domain=
=20
per request the domains won't get confused - the correct res object will be=
=20
available when an error is thrown?
BTW: I didn't mean to imply that the domain code inside node could do this=
=20
without wrapping EventEmitters or something similar. Just that (a) it=20
should do so behind-the-scenes and (b) the wrapping shouldn't take the form=
=20
of binding the EventEmitter to whatever domain happens to be active when=20
the emitter was created.
F
>
------=_Part_1121_5565958.1352932101021
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable
<br>On Wednesday, 14 November 2012 22:04:15 UTC, Forrest L Norvell wrote:<=
blockquote class=3D"gmail_quote" style=3D"margin: 0;margin-left: 0.8ex;bord=
er-left: 1px #ccc solid;padding-left: 1ex;">
<div>
<br>
</div>
<div><div><br></div><div style=3D"font-size:10pt;font-famil=
y:Helvetica"><div style=3D"color:rgb(0,0,0);font-style:normal;font-variant:=
normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-ali=
gn:-webkit-auto;text-indent:0px;text-transform:none;white-space:normal;word=
-spacing:0px;font-family:'Lucida Grande';font-size:14px"><div style=3D"colo=
r:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;lette=
r-spacing:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px=
;text-transform:none;white-space:normal;word-spacing:0px;font-size:16px;fon=
t-family:'Lucida Grande';min-height:15px"><div style=3D"color:rgb(0,0,0);fo=
nt-style:normal;font-variant:normal;font-weight:normal;letter-spacing:norma=
l;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-transform=
:none;white-space:normal;word-spacing:0px;font-family:'Lucida Grande';font-=
size:14px;min-height:16px">Forrest L Norvell</div><div style=3D"color:rgb(0=
,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spaci=
ng:normal;line-height:normal;text-align:-webkit-auto;text-indent:0px;text-t=
ransform:none;white-space:normal;word-spacing:0px;font-family:'Lucida Grand=
e';font-size:14px;min-height:16px">Node.js agent engineer</div><div style=
=3D"color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:norm=
al;letter-spacing:normal;line-height:normal;text-align:-webkit-auto;text-in=
dent:0px;text-transform:none;white-space:normal;word-spacing:0px;font-famil=
y:'Lucida Grande';font-size:12px">| E <a href=3D"javascript:" sty=
le=3D"color:rgb(0,85,103)" target=3D"_blank" gdf-obfuscated-mailto=3D"7kKZs=
ln7EGcJ">for...@newrelic.com</a> | C (<wbr>415) 823=E2=
=80=916356 | T @<a href=3D"http://twitter.com/othiym23" style=3D"=
color:rgb(43,136,154)" target=3D"_blank">othiym23</a> | G <a=
href=3D"https://github.com/othiym23" style=3D"color:rgb(150,204,204)" targ=
et=3D"_blank">github.com/<wbr>othiym23</a> | W <a href=3D"ht=
tp://newrelic.com/" style=3D"color:rgb(43,136,154)" target=3D"_blank">newre=
lic.com</a></div><div style=3D"color:rgb(0,0,0);font-family:Times;font-styl=
e:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-=
height:normal;text-align:-webkit-auto;text-indent:0px;text-transform:none;w=
hite-space:normal;word-spacing:0px;font-size:12px;margin:0px;min-height:15p=
x"><span style=3D"font-family:'Lucida Grande';font-size:14px"><span style=
=3D"color:rgb(0,85,103)"><b>(</b></span> <span style=3D"color:rgb(43,1=
36,154)"><b>(</b></span> <span style=3D"color:rgb(150,204,204)">=E2=80=
=A2</span><span style=3D"color:rgb(43,136,154)"><b>)</b></span><span style=
=3D"color:rgb(0,85,103)"><b>)</b></span> <b>New Relic</b></span>=
</div></div></div></div></div>
=20
<p style=3D"color:#a0a0a8">On Wednesday, November 14, 2012 =
at 1:35 PM, pie_lard wrote:</p>
<blockquote type=3D"cite" style=3D"border-left-style:solid;=
border-width:1px;margin-left:0px;padding-left:10px">
<span><div>Suppose node were changed to remove the feat=
ure that lets domains track EventEmitters. So domain.add(), .remove()=
and .members would all be removed. The code that automatically adds =
new EventEmitters to the active domain (if there is one) would also be dele=
ted. Finally, suppose .bind() and .intercept() were also removed.</di=
v></span></blockquote><div><span style=3D"font-size:12px">As an aside, you =
should only have to call domain.{add,remove} on a handle or an emitter your=
self under very specific (and rare) conditions.</span></div><blockquote typ=
e=3D"cite" style=3D"border-left-style:solid;border-width:1px;margin-left:0p=
x;padding-left:10px"><span><div>That would just leave domain.run() - plus t=
he ability to create new domains and possibly nest them.</div><div><br></di=
v><div>Would that be enough to effectively recreate synchronous stack-based=
exception handling?</div></span></blockquote><div><span style=3D"font-size=
:12px">No, and I'll explain why below.</span> </div><blockquote type=
=3D"cite" style=3D"border-left-style:solid;border-width:1px;margin-left:0px=
;padding-left:10px"><span><div>My answer would be yes - and in fact it woul=
d remove my connection-pool problem altogether without any need for the 3rd=
-party module authors to change anything.</div><div><br></div><div>To put i=
t another way; in what real-world circumstances would you need the features=
I just removed?</div><div><br></div><div>Please understand I ask this as a=
bit of a node noob ;) It's not a criticism of domains - I would like=
someone to explain why I'm wrong about this.</div></span></blockquote><div=
><span style=3D"font-size:12px">The problem is it's impossible to know in a=
dvance whether a chunk of code is going to use EventEmitters or not, and wi=
thout domain code there to wrap them, you'd lose the necessary state to tie=
a specific request to a specific domain. Consider the following (buggy / i=
ncomplete) Express handler:</span></div><div><span style=3D"font-size:12px"=
><br></span></div><div><pre style=3D"margin-top:0px;margin-bottom:0px;paddi=
ng:0px;font-size:12px;line-height:1.4em;font-family:'Bitstream Vera Sans Mo=
no',Courier,monospace;text-align:left"><div style=3D"margin:0px;padding:0px=
0px 0px 1em;line-height:1.4em"><pre style=3D"margin-top:0px;margin-bottom:=
0px;padding:0px;line-height:1.4em;font-family:'Bitstream Vera Sans Mono',Co=
urier,monospace"><div style=3D"margin:0px;padding:0px 0px 0px 1em;line-heig=
ht:1.4em"><span style=3D"margin:0px;padding:0px;line-height:1.4em">app</spa=
n><span style=3D"margin:0px;padding:0px;line-height:1.4em">.</span><span st=
yle=3D"margin:0px;padding:0px;line-height:1.4em">get</span><span style=3D"m=
argin:0px;padding:0px;line-height:1.4em">(</span><span style=3D"margin:0px;=
padding:0px;line-height:1.4em;color:rgb(221,17,68)">'/test'</span><span sty=
le=3D"margin:0px;padding:0px;line-height:1.4em">,</span> <span style=3D"mar=
gin:0px;padding:0px;line-height:1.4em;font-weight:bold">function</span> <sp=
an style=3D"margin:0px;padding:0px;line-height:1.4em">(</span><span style=
=3D"margin:0px;padding:0px;line-height:1.4em">req</span><span style=3D"marg=
in:0px;padding:0px;line-height:1.4em">,</span> <span style=3D"margin:0px;pa=
dding:0px;line-height:1.4em">res</span><span style=3D"margin:0px;padding:0p=
x;line-height:1.4em">)</span> <span style=3D"margin:0px;padding:0px;line-he=
ight:1.4em">{</span></div><div style=3D"margin:0px;padding:0px 0px 0px 1em;=
line-height:1.4em"> <span style=3D"margin:0px;padding:0px;line-h=
eight:1.4em;font-weight:bold">var</span> <span style=3D"margin:0px;padding:=
0px;line-height:1.4em">d</span> <span style=3D"margin:0px;padding:0px;line-=
height:1.4em;font-weight:bold">=3D</span> <span style=3D"margin:0px;padding=
:0px;line-height:1.4em">domain</span><span style=3D"margin:0px;padding:0px;=
line-height:1.4em">.</span><span style=3D"margin:0px;padding:0px;line-heigh=
t:1.4em">create</span><span style=3D"margin:0px;padding:0px;line-height:1.4=
em">();</span></div><div style=3D"margin:0px;padding:0px 0px 0px 1em;line-h=
eight:1.4em"> <span style=3D"margin:0px;padding:0px;line-height:=
1.4em">d</span><span style=3D"margin:0px;padding:0px;line-height:1.4em">.</=
span><span style=3D"margin:0px;padding:0px;line-height:1.4em">on</span><spa=
n style=3D"margin:0px;padding:0px;line-height:1.4em">(</span><span style=3D=
"margin:0px;padding:0px;line-height:1.4em;color:rgb(221,17,68)">'error'</sp=
an><span style=3D"margin:0px;padding:0px;line-height:1.4em">,</span> <span =
style=3D"margin:0px;padding:0px;line-height:1.4em;font-weight:bold">functio=
n</span> <span style=3D"margin:0px;padding:0px;line-height:1.4em">(</span><=
span style=3D"margin:0px;padding:0px;line-height:1.4em">error</span><span s=
tyle=3D"margin:0px;padding:0px;line-height:1.4em">)</span> <span style=3D"m=
argin:0px;padding:0px;line-height:1.4em">{</span> <span style=3D"margin:0px=
;padding:0px;line-height:1.4em">res</span><span style=3D"margin:0px;padding=
:0px;line-height:1.4em">.</span><span style=3D"margin:0px;padding:0px;line-=
height:1.4em">send</span><span style=3D"margin:0px;padding:0px;line-height:=
1.4em">(</span><span style=3D"margin:0px;padding:0px;line-height:1.4em;colo=
r:rgb(0,153,153)">500</span><span style=3D"margin:0px;padding:0px;line-heig=
ht:1.4em">,</span> <span style=3D"margin:0px;padding:0px;line-height:1.4em"=
>{</span><span style=3D"margin:0px;padding:0px;line-height:1.4em;color:rgb(=
221,17,68)">"content-type"</span><span style=3D"margin:0px;padding:0px;line=
-height:1.4em;font-weight:bold">:</span><span style=3D"margin:0px;padding:0=
px;line-height:1.4em;color:rgb(221,17,68)">"text/plain"</span><span style=
=3D"margin:0px;padding:0px;line-height:1.4em">},</span> <span style=3D"marg=
in:0px;padding:0px;line-height:1.4em">error</span><span style=3D"margin:0px=
;padding:0px;line-height:1.4em">.</span><span style=3D"margin:0px;padding:0=
px;line-height:1.4em">stack</span><span style=3D"margin:0px;padding:0px;lin=
e-height:1.4em">);</span> <span style=3D"margin:0px;padding:0px;line-height=
:1.4em">});</span></div><div style=3D"margin:0px;padding:0px 0px 0px 1em;li=
ne-height:1.4em"> <span style=3D"margin:0px;padding:0px;line-hei=
ght:1.4em">d</span><span style=3D"margin:0px;padding:0px;line-height:1.4em"=
>.</span><span style=3D"margin:0px;padding:0px;line-height:1.4em">run</span=
><span style=3D"margin:0px;padding:0px;line-height:1.4em">(</span><span sty=
le=3D"margin:0px;padding:0px;line-height:1.4em;font-weight:bold">function</=
span> <span style=3D"margin:0px;padding:0px;line-height:1.4em">()</span> <s=
pan style=3D"margin:0px;padding:0px;line-height:1.4em">{</span></div><div s=
tyle=3D"margin:0px;padding:0px 0px 0px 1em;line-height:1.4em"> &=
nbsp; <span style=3D"margin:0px;padding:0px;line-height:1.4em">fs</spa=
n><span style=3D"margin:0px;padding:0px;line-height:1.4em">.</span><span st=
yle=3D"margin:0px;padding:0px;line-height:1.4em">readFile</span><span style=
=3D"margin:0px;padding:0px;line-height:1.4em">(</span><span style=3D"margin=
:0px;padding:0px;line-height:1.4em">FILENAME</span><span style=3D"margin:0p=
x;padding:0px;line-height:1.4em">,</span> <span style=3D"margin:0px;padding=
:0px;line-height:1.4em;color:rgb(221,17,68)">'utf-8'</span><span style=3D"m=
argin:0px;padding:0px;line-height:1.4em">,</span> <span style=3D"margin:0px=
;padding:0px;line-height:1.4em;font-weight:bold">function</span> <span styl=
e=3D"margin:0px;padding:0px;line-height:1.4em">(</span><span style=3D"margi=
n:0px;padding:0px;line-height:1.4em">err</span><span style=3D"margin:0px;pa=
dding:0px;line-height:1.4em">,</span> <span style=3D"margin:0px;padding:0px=
;line-height:1.4em">contents</span><span style=3D"margin:0px;padding:0px;li=
ne-height:1.4em">)</span> <span style=3D"margin:0px;padding:0px;line-height=
:1.4em">{</span></div><div style=3D"margin:0px;padding:0px 0px 0px 1em;line=
-height:1.4em"> <span style=3D"margin:0p=
x;padding:0px;line-height:1.4em;color:rgb(153,153,136);font-style:italic">/=
/ process.domain should be available here</span></div><div style=3D"margin:=
0px;padding:0px 0px 0px 1em;line-height:1.4em"> &nbs=
p; <span style=3D"margin:0px;padding:0px;line-height:1.4em">res</span>=
<span style=3D"margin:0px;padding:0px;line-height:1.4em">.</span><span styl=
e=3D"margin:0px;padding:0px;line-height:1.4em">send</span><span style=3D"ma=
rgin:0px;padding:0px;line-height:1.4em">(</span><span style=3D"margin:0px;p=
adding:0px;line-height:1.4em">contents</span><span style=3D"margin:0px;padd=
ing:0px;line-height:1.4em">.</span><span style=3D"margin:0px;padding:0px;li=
ne-height:1.4em">replac<wbr>e</span><span style=3D"margin:0px;padding:0px;l=
ine-height:1.4em">(</span><span style=3D"margin:0px;padding:0px;line-height=
:1.4em;color:rgb(0,153,38)">/:/gm</span><span style=3D"margin:0px;padding:0=
px;line-height:1.4em">,</span> <span style=3D"margin:0px;padding:0px;line-h=
eight:1.4em;color:rgb(221,17,68)">'/'</span><span style=3D"margin:0px;paddi=
ng:0px;line-height:1.4em">));</span></div><div style=3D"margin:0px;padding:=
0px 0px 0px 1em;line-height:1.4em"> <span style=3D"m=
argin:0px;padding:0px;line-height:1.4em">});</span></div><div style=3D"marg=
in:0px;padding:0px 0px 0px 1em;line-height:1.4em"> <span style=
=3D"margin:0px;padding:0px;line-height:1.4em">});</span></div><div style=3D=
"margin:0px;padding:0px 0px 0px 1em;line-height:1.4em"><span style=3D"margi=
n:0px;padding:0px;line-height:1.4em">});</span></div></pre></div></pre></di=
v><div><span style=3D"font-size:12px"><br></span></div><div><span style=3D"=
font-size:12px">The call to res.send is going to happen several passes=
through the event loop after the initial handler is called. It's entirely =
possible that in the time the file is read from disk, another several reque=
sts to that handler are going to have come in to the server. Without dealin=
g with the EventEmitters, how will any errors emitted inside the fs.readFil=
e callback be connected back to the enclosing domain?</span></div><div><spa=
n style=3D"font-size:12px"><br></span></div></blockquote><div>Maybe this is=
what I'm misunderstanding! I was under the impression that your exam=
ple would in fact work (ie. that your domain handler would catch an error t=
hrown by the res.send() call).</div><div><br></div><div>The docs say:</div>=
<div><br></div><div><span style=3D"color: rgb(0, 0, 0); font-family: Georgi=
a, FreeSerif, Times, serif; font-size: 16px; line-height: 22px; ">Additiona=
lly, callbacks passed to lowlevel event loop requests (such as to fs.open, =
or other callback-taking methods) will automatically be bound to the active=
domain. If they throw, then the domain will catch the error.</span><br></d=
iv><div><br></div><div>Doesn't that imply that you've done all you need to =
do in the above code? That just by starting the chain of callbacks in=
side run() that the active domain will persist through them? And that=
as you're creating a new domain per request the domains won't get confused=
- the correct res object will be available when an error is thrown?</div><=
div><br></div><div>BTW: I didn't mean to imply that the domain code inside =
node could do this without wrapping EventEmitters or something similar. &nb=
sp;Just that (a) it should do so behind-the-scenes and (b) the wrapping sho=
uldn't take the form of binding the EventEmitter to whatever domain happens=
to be active when the emitter was created.</div><div><br></div><div><br></=
div><blockquote class=3D"gmail_quote" style=3D"margin: 0;margin-left: 0.8ex=
;border-left: 1px #ccc solid;padding-left: 1ex;"><div><span style=3D"font-s=
ize:12px"></span></div><div><span style=3D"font-size:12px">F</span></div>
</blockquote>
------=_Part_1121_5565958.1352932101021--
------=_Part_1120_19438936.1352932101014--
Message from discussion Domains and connection pools
| Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy |
| ©2013 Google |