force coroutine to execute on seperate os thread

[PPe]

Hi Go community!

i've started using go recently and find it very nice,

i am however having a problem with the cairo (vectorial drawing) api
it is supposed to be threadsafe
i generate many pictures from data out of a simulation
to do this i wanted to start a certain number a coroutine to draw to
separate drawing context

however i find that with multiple coroutines drawing, at some point
the api segfaults
i believe it might have to do with some context switching of the api
i found that it is possible to lock os threads with
runtime.LockOSThread()
probably useful when cairo calls opengl
but it doesnt solves my problem

is there a way i can force execution of the coroutines doing the
drawing on separate OS threads?
i was thinking maybe that would solve my problem

many thanks

PL

[Russ Cox]

It sounds like something you are doing in Cairo
cannot be done from multiple threads simultaneously.
I did a search for [cairo threadsafe] and found various
email threads suggesting that as long as you are really
using different cairo objects in the different threads
you should be okay. But maybe that's wrong or maybe
you are using the same cairo object from multiple
threads (here, approximately, thread == goroutine).

runtime.LockOSThread is useful if you need all calls
to an API to originate in a single thread. This is necessary
for some brain-dead APIs like SDL and NaCl and ptrace
that maintain hidden internal per-thread state.
That doesn't sound like the problem here.

I would start by just introducing a top level

var l sync.Mutex

and then around each call into cairo do

l.Lock()
... cairo stuff ...
l.Unlock()

and see if that fixes the problem. That will make
only one goroutine at a time be able to call into cairo,
which is very likely to keep it from crashing, at the cost
of not having any parallelism. Assuming that fixes the
problem, then you could start removing the locking one
call site at a time and see when things begin to break again.

Russ

[Philippe Leroux]

many thanks for you ideas and research time

i didnt  mention, but it is apparently when i call cairo_stroke() that it segfaults

(i posted first on cairo mailing list but no reply)

without calling this function it works (but without any outputs ...)

whoever, each go routines that draws, creates its own cairo objects and draw to them independently (and using different data sets, no concurrency issue here). so as far as i understood cairo multi-threaded issue, it shouldn't be a problem 

i have tried mutex, but i loose so much time with mutex as each call to stroke takes a lot of time (that's basically where the drawing is applied to the drawing surface) it's almost (completely)  as if not using multithread power..

that's the reason why i was thinking maybe the problem is in one os thread, on which multiple goroutines are multiplexed,

a call to stroke by one goroutine would be conflicting with another call from another routine whereby cairo is creating some status variables per thread (and of course not per go routines)

or maybe i misunderstood something there

if i'm correct, and if it is possible to force a goroutine to be executed on an independant (not multiplexed) thread, than that would solve my problem

please correct me if i  misunderstood the workings of goroutine

many thanks!

[Russ Cox]

> that's the reason why i was thinking maybe the problem is in one os thread,
> on which multiple goroutines are multiplexed,
> a call to stroke by one goroutine would be conflicting with another call
> from another routine whereby cairo is creating some status variables per
> thread (and of course not per go routines)
> or maybe i misunderstood something there
> if i'm correct, and if it is possible to force a goroutine to be executed on
> an independant (not multiplexed) thread, than that would solve my problem

it sounds like you want runtime.LockOSThread.
after a goroutine has called LockOSThread, it is
guaranteed that that goroutine will always execute
in that thread, and no other goroutine will.

russ

[Scott Pakin]

On Dec 14, 10:56 am, Russ Cox <r...@golang.org> wrote:
> it sounds like you want runtime.LockOSThread.
> after a goroutine has called LockOSThread, it is
> guaranteed that that goroutine will always execute
> in that thread, and no other goroutine will.

If there's only one OS thread at the time runtime.LockOSThread is
called, will this block other goroutines from running, or will the Go
runtime spawn another OS thread to process the remaining goroutines?

[Ian Lance Taylor]

Scott Pakin <scot...@pakin.org> writes:

The latter. A new OS thread will be spawned as needed.

Ian

[Philippe Leroux]

Ah!! ok

i did not understood how that worked

i did call LockOS Thread in my main function and thought that was all that was needed

so now i call it at the begining of the drawing function which is "forked" by "go"

....

and

....

segfault

oh dammit, i really was hopping that would do the trick this time

no luck... still segfaults inside some ati libGL called by cairo (does the same with mesa gl though)

so i guess, it is up to cairo mailing list now?

although sometimes i dont get error inside cairo/gl but i get

panic: runtime error: SIGNONE: no trap

from go i believe? if you have any ideas about that. i've searched the mailing list and found an entry about some error in go that was corrected. so i'm not sure where to look for that 

many thanks for clarifying me how lockos works. i must have read the documentation too fast or something

sincerely

PL

[Philippe Leroux]

oops forgot to reply all, here it is

---------- Forwarded message ----------
From: Philippe Leroux <philippe.la...@gmail.com>
Date: Wed, Dec 15, 2010 at 2:29 PM
Subject: Re: [go-nuts] force coroutine to execute on seperate os thread
To: r...@golang.org

so from cairo, they are working on a atomic operation version of their pixman library (which rights to the surface)

and with this version

my code does not segfault with this 

they told me that yes, the pixman library requires one surface per thread

however, i wonder if i have still something wrong forcing a goroutine  to be on a single os thread

as, that is what my code  (is supposed to? ) do

one (go) drawing function (function called via keyword go) first calls lockOSthreads, creates a surface, draws to it, saves image ....

could i still have done something wrong with starting my goroutine to be locked on thread?

does lockosthread reserves that thread to this goroutine?

or does it just locks that goroutine to that thread (allowing multiplexing with other goroutines)?

many thanks

Philippe

[roger peppe]

maybe you could post a small amount of code that demonstrates
the problem?

On 15 December 2010 13:30, Philippe Leroux

[Philippe Leroux]

to correct mysefl,

cairo and underlying libpixman does not require one surface per thread

but one surface to be used only from one thread.

i've tried another solution though, that does not require the libpixman modified atomic operations

and it works, 

apparently is a "strong hint that you're having problems with atomic refcounting."

so i'll develop that on the cairo mailing list

and come back to report here

as a note for people looking through this thread for solutions on multithreaded cairo follow this thread:

http://lists.cairographics.org/archives/cairo/2010-December/021288.html

many thanks

[Philippe Leroux]

my code  (important parts)):

//i init here the number of threads i want to draw

//this is called from main, at start

//the sentData channel holds pointers to data sets that are unused

func Init(MobileNumber int, ConnectionNumber int) {

sentData = make(chan *DataSave, numImThread)

for i := 0; i < numImThread; i++ {

//go drawP( MobileNumber, ConnectionNumber)

go DrawVoronoi( MobileNumber, ConnectionNumber)

}

}

// in this  thread, i lock thread, i create a data set that holds the data to be drawn 

// as well as a cairo surface and context to draw to (in d.Init)

// i sent that data pointer to some thread which synchronizes things 

//    - copies data to the dataset and when that is done

//    - sends an acknoledge on ackChan to start the drawaing 

// the drawing thread gets the ack, starts to draw

// sends back the dataset to get some new data to draw

func DrawVoronoi(MobileNumber, ConnectionNumber int){

runtime.LockOSThread() 

d := new(DataSave)

d.Init(MobileNumber, ConnectionNumber)

sentData <- d // we are ready to draw

for _ = range d.ackChan {

drawIt(d)

         sentData <- d //we are done

}

}

// this function called by main, when all is synchronised to copie data safely before it gets modified again

// waits for a dataset to copy data to, copies data, send acknoledge 

func Draw(syns []synstation.DBS, mobs []synstation.Mob, k int) {

data := <-sentData

for i := range whateverdataToCopy {

// copies data here to data set "data"

}

data.ackChan <- 1

}

the data set struct contains the ack channel, data to draw, and drawing stuct

type DataSave struct {

k       int

mobs    []synstation.EmitterS

connec  []synstation.ConnectionS

NumConn int

ackChan chan int

cv gocairo.Canvas

}

[roger peppe]

On 15 December 2010 14:28, Philippe Leroux
<philippe.la...@gmail.com> wrote:
> my code  (important parts)):

a complete example would be better (reduced
as much as you can).

for instance, it's not clear exactly where you're
calling cairo.