SPAWN command failures on Apple M1 processor

[patric...@icloud.com]

If you run IDL on an Apple M1 processor, you may find that calls to SPAWN sometimes produces hard errors of the following form:

  tcsh(30796,0x2046f0e00) malloc: can't allocate region

  :*** mach_vm_map(size=1048576, flags: 100) failed (error code=268435465)

  tcsh(30796,0x2046f0e00) malloc: *** set a breakpoint in malloc_error_break to debug

  Out of memory

As you can see above, the shell created by SPAWN (tcsh in my case) is the process failing (not IDL).  This failure is random/intermittent.

M1 machines currently contain both Intel and ARM compilations of some tools, including /bin/tcsh  (i.e. "universal binaries").  The failure above has been demonstrated outside the context of IDL; see  https://openradar.appspot.com/FB9666962

That author triggered the failure by forcing an M1 machine (which uses the ARM architecture) to run the Intel version of /bin/csh

On an M1 machine, normal shells that you interact with are running the ARM compilation, which you can demonstrate as follows:

% arch; uname -p

arm64

arm

However, shells created by IDL's SPAWN command are running the Intel compilation, which you can demonstrate as follows:

IDL> spawn, 'arch; uname -p'

i386

i386

Presumably, this occurs because (currently) IDL itself is an Intel compilation.

One way to work-around this problem is to revise your SPAWN calls to use the "arch" command to force the spawned shells to use their ARM compilation, e.g.

IDL> spawn, /NOSHELL, ['arch', '-arm64', '/bin/tcsh', '-c', 'arch; uname -p']

arm64

arm

Presumably, once an ARM compilation of IDL is available this issue will go away.

Patrick Broos

[Wayne Landsman]

I am going to piggyback on this post to report a minor Spawn problem I am having with IDL on Linux, even though it is unrelated to the problem that Patrick reported. 
After we upgraded from IDL 8.5.1 to IDL 8.8 on Linux, I could no longer spawn to the nedit editor.      In a fresh IDL session I get 

IDL> $nedit test.txt

X Error of failed request:  BadAlloc (insufficient resources for operation)

  Major opcode of failed request:  53 (X_CreatePixmap)

  Serial number of failed request:  2742

  Current serial number in output stream:  2762

The nedit call works fine outside of IDL, but I haven't had problems in IDL with Spawning to any other Linux command (including gedit).    So I just keep a separate open terminal window for using the nedit editor.

--Wayne

[patric...@icloud.com]

Below is code that should reproduce this problem on any M1 machine:

idl

command = 'date'

.run

for ii=0, long(1e4) do begin

  spawn, command, EXIT_STATUS=exit_status, result, /STDERR

  if (exit_status NE 0) then begin

    print, 'With-shell failed on iteration ', ii

    forprint, result

  endif

  spawn, /NOSHELL, ['arch','-arm64', command], EXIT_STATUS=exit_status, result, /STDERR

  if (exit_status NE 0) then begin

    print, 'NOSHELL failed on iteration ', ii

    forprint, result

  endif

endfor

end

On my machine, with tcsh as my default shell, about half of the loop iterations produce the "Out of memory" failure from the "With-shell" spawn (where the shell is running its i386 binary).

Patrick