Go + Imagemagick 7 + tight loop = CRASH!

[Robert Schaaf]

Please help, I humbly beg!

First, the environment:

go version go1.16.5 darwin/arm64

ImageMagick 7.0.11-8 Q16 arm 2021-04-17

macOS Big Sur, version 11.4 on Mac Mini M1, 16 GB

I've run into a brick wall with Go driving the Imagemagick 7 API on a 16

GB M1 Mac Mini. The problem occurs when combining an image read and

pixel extraction in a tight loop. The first two lines, which cure the problem, 

were suggested by Justin Israel. Note: it doesn't always crash, but the attached

PNG fails 100% of the time.

Thanks for looking.

The code looks like this:

package main

import (

"fmt"

"runtime"

"gopkg.in/gographics/imagick.v3/imagick"

)

func main() {

// runtime.LockOSThread()

// defer runtime.UnlockOSThread()

var my_grayscale = "/Path/to/your/8bit/grayscale.jpg"

// use a large size image to stress test, perhaps 1500x1500, any supported format.

var err error

imagick.Initialize()

defer imagick.Terminate()

for i := 1; i < 5001; i++ {

outline := imagick.NewMagickWand()

err = outline.ReadImage(my_grayscale)

if err != nil {

panic(err)

}

width := outline.GetImageWidth()

height := outline.GetImageHeight()

pix, err := outline.ExportImagePixels(0, 0, width, height, "I", imagick.PIXEL_CHAR)

if err != nil {

panic(err)

}

pixels, ok := pix.([]byte)

if !ok {

panic("Pixels not extracted")

}

fmt.Printf("%d: %d pixels.\n", i, len(pixels))

outline.Destroy()

}

fmt.Println("Done")

}

The crashlog and backtrace follow.

goroutine 0 [idle]:

runtime: unknown pc 0x185100e68

stack: frame={sp:0x172f4e890, fp:0x0} stack=[0x172ecf338,0x172f4ef38)

0000000172f4e790:  0000000102650400  0000000000000081 

0000000172f4e7a0:  0000000172f4e7e0  773280018514fd14 

0000000172f4e7b0:  3fd3333333333333  0000000000000000 

0000000172f4e7c0:  0000000000000000  000000010265a128 

0000000172f4e7d0:  00000001026494a8  0000000172f4e970 

0000000172f4e7e0:  0000000172f4e8e0  b23e00018514fc34 

0000000172f4e7f0:  0000000132e11420  0000000000000005 

0000000172f4e800:  3fd3333333333333  0000000000000000 

0000000172f4e810:  3feeddf5de1dde88  0000000000000000 

0000000172f4e820:  3ff5f39f935568db  0000000000000000 

0000000172f4e830:  3fdb01b18088d6a0  0000000000000000 

0000000172f4e840:  0706050403020100  0f0e0d0c0b0a0908 

0000000172f4e850:  00000001e404fa08  00000001f045e8b0 

0000000172f4e860:  000000018503dde4  0000000000000000 

0000000172f4e870:  0000000172f4e8b0  f1520001850407e8 

0000000172f4e880:  000000013701c200  0000000000000000 

0000000172f4e890: <0000000000000000  000000010265a128 

0000000172f4e8a0:  00000001026494a8  0000000172f4e8d8 

0000000172f4e8b0:  0000000172f4e8f0  fc6100018507b460 

0000000172f4e8c0:  0000000132e11420  ffffffff00000024 

0000000172f4e8d0:  0000000000000001  00000001fffff9df 

0000000172f4e8e0:  00000001026494a8  0000000172f4e970 

0000000172f4e8f0:  0000000172f4e900  713e8001025fc30c 

0000000172f4e900:  0000000172f4e930  00000001025faec4 

0000000172f4e910:  0000000000000000  0000000000000000 

0000000172f4e920:  0000000000000000  0000000172f4e940 

0000000172f4e930:  0000000172f4e9b0  00000001025f8f34 

0000000172f4e940:  0000000172f4e970  0000000172f4e950 

0000000172f4e950:  0000000000000000  0000000000000000 

0000000172f4e960:  0000000000000000  0000000000000000 

0000000172f4e970:  0000000100000005  0000000132e11bf0 

0000000172f4e980:  0000000000000134  0000000d00000004 

runtime: unknown pc 0x185100e68

stack: frame={sp:0x172f4e890, fp:0x0} stack=[0x172ecf338,0x172f4ef38)

0000000172f4e790:  0000000102650400  0000000000000081 

0000000172f4e7a0:  0000000172f4e7e0  773280018514fd14 

0000000172f4e7b0:  3fd3333333333333  0000000000000000 

0000000172f4e7c0:  0000000000000000  000000010265a128 

0000000172f4e7d0:  00000001026494a8  0000000172f4e970 

0000000172f4e7e0:  0000000172f4e8e0  b23e00018514fc34 

0000000172f4e7f0:  0000000132e11420  0000000000000005 

0000000172f4e800:  3fd3333333333333  0000000000000000 

0000000172f4e810:  3feeddf5de1dde88  0000000000000000 

0000000172f4e820:  3ff5f39f935568db  0000000000000000 

0000000172f4e830:  3fdb01b18088d6a0  0000000000000000 

0000000172f4e840:  0706050403020100  0f0e0d0c0b0a0908 

0000000172f4e850:  00000001e404fa08  00000001f045e8b0 

0000000172f4e860:  000000018503dde4  0000000000000000 

0000000172f4e870:  0000000172f4e8b0  f1520001850407e8 

0000000172f4e880:  000000013701c200  0000000000000000 

0000000172f4e890: <0000000000000000  000000010265a128 

0000000172f4e8a0:  00000001026494a8  0000000172f4e8d8 

0000000172f4e8b0:  0000000172f4e8f0  fc6100018507b460 

0000000172f4e8c0:  0000000132e11420  ffffffff00000024 

0000000172f4e8d0:  0000000000000001  00000001fffff9df 

0000000172f4e8e0:  00000001026494a8  0000000172f4e970 

0000000172f4e8f0:  0000000172f4e900  713e8001025fc30c 

0000000172f4e900:  0000000172f4e930  00000001025faec4 

0000000172f4e910:  0000000000000000  0000000000000000 

0000000172f4e920:  0000000000000000  0000000172f4e940 

0000000172f4e930:  0000000172f4e9b0  00000001025f8f34 

0000000172f4e940:  0000000172f4e970  0000000172f4e950 

0000000172f4e950:  0000000000000000  0000000000000000 

0000000172f4e960:  0000000000000000  0000000000000000 

0000000172f4e970:  0000000100000005  0000000132e11bf0 

0000000172f4e980:  0000000000000134  0000000d00000004 

goroutine 1 [syscall]:

runtime.cgocall(0x102371528, 0x14000051d38, 0x14000000100)

        /usr/local/go/src/runtime/cgocall.go:154 +0x50 fp=0x14000051d00 sp=0x14000051cc0 pc=0x1022d5ad0

gopkg.in/gographics/imagick.v3/imagick._Cfunc_MagickExportImagePixels(0x137008200, 0x0, 0x0, 0x614, 0x639, 0x132e11020, 0x100000001, 0x140003de000, 0x0)

        _cgo_gotypes.go:4595 +0x44 fp=0x14000051d30 sp=0x14000051d00 pc=0x10236bcc4

gopkg.in/gographics/imagick.v3/imagick.(*MagickWand).ExportImagePixels.func2(0x1400009c000, 0x0, 0x0, 0x614, 0x639, 0x132e11020, 0x1, 0x140003de000, 0x14000052000)

        /Users/rwschaaf/go/src/gopkg.in/gographics/imagick.v3/imagick/magick_wand_image.go:799 +0xa4 fp=0x14000051d90 sp=0x14000051d30 pc=0x10236dbd4

gopkg.in/gographics/imagick.v3/imagick.(*MagickWand).ExportImagePixels(0x1400009c000, 0x0, 0x0, 0x614, 0x639, 0x10237ae0c, 0x1, 0x1, 0x0, 0x0, ...)

        /Users/rwschaaf/go/src/gopkg.in/gographics/imagick.v3/imagick/magick_wand_image.go:799 +0x288 fp=0x14000051e90 sp=0x14000051d90 pc=0x10236ce08

main.main()

        /Users/rwschaaf/go/src/github.com/rwschaaf/stresstest/main.go:28 +0x190 fp=0x14000051f70 sp=0x14000051e90 pc=0x10236e3a0

runtime.main()

        /usr/local/go/src/runtime/proc.go:225 +0x26c fp=0x14000051fd0 sp=0x14000051f70 pc=0x102304e1c

runtime.goexit()

        /usr/local/go/src/runtime/asm_arm64.s:1130 +0x4 fp=0x14000051fd0 sp=0x14000051fd0 pc=0x102332274

r0      0x0

r1      0x0

r2      0x0

r3      0x0

r4      0x132e11420

r5      0x20

r6      0x0

r7      0x140003de000

r8      0xffddb2351033c6dc

r9      0xffddb23462c736dc

r10     0x3ffe

r11     0x1fbf51702

r12     0x1fbf51702

r13     0x1e

r14     0x881

r15     0x8000001f

r16     0x148

r17     0x1f3a8ae90

r18     0x0

r19     0x6

r20     0x172f4f000

r21     0x1b03

r22     0x172f4f0e0

r23     0x0

r24     0x13701c200

r25     0x132e11030

r26     0x140003de000

r27     0x10265a240

r28     0x0

r29     0x172f4e8b0

lr      0x18513343c

sp      0x172f4e890

pc      0x185100e68

fault   0x185100e68

[Ian Lance Taylor]

On Fri, Jun 11, 2021 at 1:32 PM Robert Schaaf <rwsc...@gmail.com> wrote:
>
> Please help, I humbly beg!
>
> First, the environment:
>
> go version go1.16.5 darwin/arm64
> ImageMagick 7.0.11-8 Q16 arm 2021-04-17
> macOS Big Sur, version 11.4 on Mac Mini M1, 16 GB
>
> I've run into a brick wall with Go driving the Imagemagick 7 API on a 16
> GB M1 Mac Mini. The problem occurs when combining an image read and
> pixel extraction in a tight loop. The first two lines, which cure the problem,
> were suggested by Justin Israel. Note: it doesn't always crash, but the attached
> PNG fails 100% of the time.

My understanding of what you are saying is that you have a way to
avoid the crash, which is to call runtime.LockOSThread. Is there
something wrong with that approach? What kind of answer are you
looking for? Thanks.

Ian

[Justin Israel]

Some important context seems to have been omitted in this thread from the original question as it was posted in the imagick issue:
https://github.com/gographics/imagick/pull/256#issuecomment-858047483

Robert said it only crashes on a Mac M1 with ImageMagick7, whereas it worked fine on a Mac amd64. And I had also tested it and found it to be working fine on Ubuntu 18.04 amd64. My suggestion of trying LockOSThread apparently fixed the crash on the M1 target. And then I suggested Robert pose the situation to golang-nuts to see if anything popped up as being specific to this newer Mac target and why it might behave differently than other targets.