Cautionary tale : Remember to close StringToStream...

[Roger Wilson]

I some code running extremely slow and finally traced the problem to a division by zero which should take very little time...

2/3/12 10:10:15 In[2]:= Timing[1.0/0.0]
2/3/12 10:10:15 During evaluation of In[2]:= Power::infy: Infinite expression 1/0. encountered. >>
2/3/12 10:10:15 Out[2]= {0.,ComplexInfinity}

However I was experiencing something like this...

2/3/12 10:11:04 In[4]:= Timing[1.0/0.0]
2/3/12 10:11:12 During evaluation of In[4]:= Power::infy: Infinite expression 1/0. encountered. >>
2/3/12 10:11:12 Out[4]= {7.75,ComplexInfinity}

As the Barman said to the Bear "Why the big paws?"...

I traced the problem to the way I was reading in my data. I had loaded a big file as a string, then having split this into separate items ran a pattern search for everything which looked like a number and then used Read[StringToStream[x], Number] to convert those items into numbers. You will note that I did not close the streams created by StringToStream.

It appears that the time taken to process Message scales approximately as the square of the number of open streams. Not a problem with say 1000 streams open but with 30,000 streams left open, on my machine, results in a few seconds of delay. With hundreds of thousands of open streams my code just stopped.

Turning the warning message off...
2/3/12 10:15:43 In[5]:= Off[Power::infy]; Timing[1.0/0.0]
2/3/12 10:15:43 Out[5]= {0.,ComplexInfinity}
Sidesteps the problem, so it appears that it is a stream issue which makes some kind of sense as Messages come down a stream of their own.

In the future I will be more careful but this amused me (once I'd stopped bashing my head into the desk) so I thought I'd share it.

ReadClose[str_String]:=Reap[Apply[(Sow[Read[#,Number]]; Close[#])&,{StringToStream[str]}]][[2,1,1]];