I recently received a note from a user saying that my procedure readcol.pro ( http://idlastro.gsfc.nasa.gov/ftp/pro/misc/readcol.pro ) - one of many such procedures to read an ASCII file into IDL variables -- was taking ~100 times longer to run from the IDL workbench than from the IDL command prompt. I of course replied that this was complete nonsense, and that the interface used to call the procedure shouldn't affect its speed.
Of course, when I tested this, a run that took under 1 second from the command prompt completely hung my Mac (x86_64 darwin unix Mac OS X 7.1 Apr 21 2009 64 64) when run from the Workbench. My Linux box was not quite as bad but also eventually hung with messages such as
java.lang.OutOFMemoryError: unable to create new native thread java.lang.ArrayIndexOutOfBoundsException: 2601
The culprit apparently is my use of SCOPE_VARFETCH. readcol.pro calls SCOPE_VARFETCH thousands of time for a large file, since it writes data directly into the output variables, and uses a SCOPE_VARFETCH call for each item written.
Below is a little test program to be called with IDL> test, x1,x2,x3,x4,x5,x6,x7,x8,x9
I find it runs instantly from the IDL command line but takes very long or hangs the IDL workbench. --Wayne
pro test,a,b,c,d,e,f,g,h,j
vv = ['a','b','c','d','e','f','g','h','j'] for k=0,8 do begin ;Create output variables res = execute(vv[k] + '=fltarr(5000)' ) endfor for jj=0,4999 do for k=0,8 do (scope_varfetch(vv[k],Level=0))[jj]= float(jj) return end
wlandsman writes: > Below is a little test program to be called with > IDL> test, x1,x2,x3,x4,x5,x6,x7,x8,x9
> I find it runs instantly from the IDL command line but takes very long > or hangs the IDL workbench. --Wayne
Yikes! It broke my Windows machine running IDL 7.1 from the Workbench. JAVA out of memory error.
Cheers,
David
-- David Fanning, Ph.D. Fanning Software Consulting, Inc. Coyote's Guide to IDL Programming: http://www.dfanning.com/ Sepore ma de ni thui. ("Perhaps thou speakest truth.")
On Wed, 18 Nov 2009, wlandsman wrote: > Of course, when I tested this, a run that took under 1 second from the > command prompt completely hung my Mac (x86_64 darwin unix Mac OS X > 7.1 Apr 21 2009 64 64) when run from the Workbench. My Linux box > was not quite as bad but also eventually hung with messages such as
> java.lang.OutOFMemoryError: unable to create new native thread > java.lang.ArrayIndexOutOfBoundsException: 2601
> The culprit apparently is my use of SCOPE_VARFETCH. readcol.pro > calls SCOPE_VARFETCH thousands of time for a large file, since it > writes data directly into the output variables, and uses a > SCOPE_VARFETCH call for each item written.
> Below is a little test program to be called with > IDL> test, x1,x2,x3,x4,x5,x6,x7,x8,x9
> I find it runs instantly from the IDL command line but takes very long > or hangs the IDL workbench. --Wayne
> pro test,a,b,c,d,e,f,g,h,j
> vv = ['a','b','c','d','e','f','g','h','j'] > for k=0,8 do begin ;Create output variables > res = execute(vv[k] + '=fltarr(5000)' ) > endfor > for jj=0,4999 do for k=0,8 do (scope_varfetch(vv[k],Level=0))[jj]=float(jj) > return > end
There is no time difference between the CLI and GUI runs in IDL 7.0 on 64 bit linux. (It's a fresh openSUSE 11.2 machine with the OpenJDK 1.6 java environment.)
Temporary buffers will solve your problem:
pro test,a,b,c,d,e,f,g,h,j
vv = ['a','b','c','d','e','f','g','h','j']
ptmp=ptrarr(9) for k=0,8 do begin ;Create output variables ptmp[k] = ptr_new(fltarr(5000)) endfor
for jj=0l,4999 do for k=0,8 do (*(ptmp[k]))[jj]=float(jj)
for k=0,8 do begin (scope_varfetch(vv[k],Level=0))=temporary(*(ptmp[k])) endfor
On Nov 18, 2:01 pm, FÖLDY Lajos <fo...@rmki.kfki.hu> wrote:
> There is no time difference between the CLI and GUI runs in IDL 7.0 on 64 > bit linux. (It's a fresh openSUSE 11.2 machine with the OpenJDK 1.6 java > environment.)
I also find no problem with IDL 7.0 on Red Hat 3.4.5-2. So most likely this is a bug introduced in IDL 7.1.
> Temporary buffers will solve your problem:
Well, its a workaround for the problem (and appears to be somewhat faster even on the command line). But it would still be nice to get the SCOPE_VARFETCH bug fixed.
> I recently received a note from a user saying that my procedure > readcol.pro > (http://idlastro.gsfc.nasa.gov/ftp/pro/misc/readcol.pro) - one of > many such procedures to read an ASCII file into IDL variables -- was > taking ~100 times longer to run from the IDL workbench than from the > IDL command prompt. I of course replied that this was complete > nonsense, and that the interface used to call the procedure shouldn't > affect its speed.
> Of course, when I tested this, a run that took under 1 second from the > command prompt completely hung my Mac (x86_64 darwin unix Mac OS X > 7.1 Apr 21 2009 64 64) when run from the Workbench. My Linux box > was not quite as bad but also eventually hung with messages such as
> java.lang.OutOFMemoryError: unable to create new native thread > java.lang.ArrayIndexOutOfBoundsException: 2601
> The culprit apparently is my use of SCOPE_VARFETCH. readcol.pro > calls SCOPE_VARFETCH thousands of time for a large file, since it > writes data directly into the output variables, and uses a > SCOPE_VARFETCH call for each item written.
> Below is a little test program to be called with > IDL> test, x1,x2,x3,x4,x5,x6,x7,x8,x9
> I find it runs instantly from the IDL command line but takes very long > or hangs the IDL workbench. --Wayne
> pro test,a,b,c,d,e,f,g,h,j
> vv = ['a','b','c','d','e','f','g','h','j'] > for k=0,8 do begin ;Create output variables > res = execute(vv[k] + '=fltarr(5000)' ) > endfor > for jj=0,4999 do for k=0,8 do (scope_varfetch(vv[k],Level=0))[jj]= > float(jj) > return > end
Very odd. It ran in the Workbench in about the same time, but after it is done, a lot of error messages get written to the terminal from which I called the Workbench, and after a few seconds, the Workbench dies with the same kind of error messages. This is with IDL 7.1 on Fedora 11 64 bit.
I tried turning off some visualizations (I though that Debug and Variables might be doing something to keep track of the call stack and local scope variables, and the problem might be there), but saw no change. I would write to ITTVIS, since this is definitely a bug.
Seems to be a bug in the IDL 7.1 workbench. The workbench craches with a lot of java errors in { x86 Win32 Windows Microsoft Windows 7.1.1 Aug 19 2009 32 64}.
Your test program runs fast and without errors in the command line version as well as in the version 7.0 workbench { x86 Win32 Windows Microsoft Windows 7.0 Oct 25 2007 32 64}.
On Nov 18, 3:07 pm, Heinz Stege <public.215....@arcor.de> wrote:
> Seems to be a bug in the IDL 7.1 workbench. The workbench craches > with a lot of java errors in { x86 Win32 Windows Microsoft Windows > 7.1.1 Aug 19 2009 32 64}.
A little bird told me that ITTVIS had discovered this bug about 3 weeks ago, and already has made the fix, which will be included in the next IDL patch release. --Wayne
On Nov 18, 7:55 pm, wlandsman <wlands...@gmail.com> wrote:
> On Nov 18, 3:07 pm, Heinz Stege <public.215....@arcor.de> wrote:> Seems to be a bug in the IDL 7.1 workbench. The workbench craches > > with a lot of java errors in { x86 Win32 Windows Microsoft Windows > > 7.1.1 Aug 19 2009 32 64}.
> A little bird told me that ITTVIS had discovered this bug about 3 > weeks ago, and already has made the fix, which will be included in the > next IDL patch release. --Wayne
The fix is CR56369 and it is now available in the IDL 7.1.2 patch release. Please note, you will need to contact ITT VIS Technical Support (TechSupp...@ittvis.com) for the link to download this patch. Be sure to mention your OS platform and architecture (32 or 64 bit). Other than this CR, this patch releas fixes an issue with extended ascii characters.
