Threading plus multiprocessing plus cv2 error

[John O'Hagan]

Dear list

Thanks to this list, I haven't needed to ask a question for
a very long time, but this one has me stumped.

Here's the minimal 3.8 code, on Debian testing:

-----
from multiprocessing import Process
from threading import Thread
from time import sleep
import cv2

def show(im, title, location):
cv2.startWindowThread()
cv2.namedWindow(title)
cv2.moveWindow(title, *location)
cv2.imshow(title, im)
sleep(2) #just to keep window open

im1 = cv2.imread('/path/to/image1')
im2 = cv2.imread('/path/to/image2')

Thread(target=show, args=(im1, 'im1', (600,0))).start()
sleep(1)
Process(target=show, args=(im2, 'im2', (0, 0))).start()
-----

Here's the error:

-----
[xcb] Unknown sequence number while processing queue
[xcb] Most likely this is a multi-threaded client and XInitThreads has
not been called
[xcb] Aborting, sorry about that.
python3: ../../src/xcb_io.c:260: poll_for_event: Assertion
`!xcb_xlib_threads_sequence_lost' failed.
-----

There's no error without the sleep(1), nor if the Process is started
before the Thread, nor if two Processes are used instead, nor if two
Threads are used instead. IOW the error only occurs if a Thread is
started first, and a Process is started a little later.

Any ideas what might be causing the error?

Thanks.

--

John

[Stephane Tougard]

On 2020-08-29, Dennis Lee Bieber <wlf...@ix.netcom.com> wrote:
> Under Linux, multiprocessing creates processes using fork(). That means
> that, for some fraction of time, you have TWO processes sharing the same
> thread and all that entails (if it doesn't overlay the forked process with
> a new executable, they are sharing the thread until the thread exits).
> same error condition even with the sleep(1) in place.

I'm not even that makes sense, how 2 processes can share a thread ?

[Chris Angelico]

They can't. However, they can share a Thread object, which is the
Python representation of a thread. That can lead to confusion, and
possibly the OP's error (I don't know for sure, I'm just positing).

ChrisA

[Karen Shaeffer]

Hello,
On linux, fork is a kernel system call. The linux kernel creates two identical processes running in separate memory spaces. At the time of creation, these memory spaces have the same content. There are some issues to be aware of. Just type ‘man fork’ on the command line of a linux system, and you can read about the issues of concern, presuming you have installed the manual pages for the linux kernel system calls.

If the forked process doesn’t overlay onto a separate memory space, then the fork system call fails, returning a failure code to the parent process. When the linux kernel is executing the fork system call, the parent (forking process) is blocked on the system call. The linux kernel actually takes over the parent process during execution of the system call, running that process in kernel mode during the execution of the fork process. The parent (forking) process only restarts, after the kernel returns.

On linux, within a given process, threads share the same memory space. If that process is the python interpreter, then the Global lock ensures only one thread is running when the fork happens. After the fork, then you have two distinct processes running in two separate memory spaces. And the fork man page discusses the details of concern with regards to specific kernel resources that could be referenced by those two distinct processes. The thread context is just a detail in that respect. All the threads of the parent process that forked the new process all share the same parent memory space.

humbly,
kls

[John O'Hagan]

On Sat, 29 Aug 2020 13:01:12 -0400

Dennis Lee Bieber <wlf...@ix.netcom.com> wrote:

> On Sat, 29 Aug 2020 18:24:10 +1000, John O'Hagan
> <rese...@johnohagan.com> declaimed the following:

>
> >There's no error without the sleep(1), nor if the Process is started
> >before the Thread, nor if two Processes are used instead, nor if two
> >Threads are used instead. IOW the error only occurs if a Thread is
> >started first, and a Process is started a little later.
> >
> >Any ideas what might be causing the error?
> >
>

> Under Linux, multiprocessing creates processes using fork().
> That means that, for some fraction of time, you have TWO processes
> sharing the same thread and all that entails (if it doesn't overlay
> the forked process with a new executable, they are sharing the thread
> until the thread exits).
>

> https://stackoverflow.com/questions/54466572/how-to-properly-multithread-in-opencv-in-2019
> (which points to)
> https://answers.opencv.org/question/32415/thread-safe/?answer=32452#post-id-32452
> """
> The library itself is thread safe in that you can have multiple calls
> into the library at the same time, however the data is not always
> thread safe. """
>
> The sleep(1), when compounded with the overhead of starting
> the thread, and then starting the process, likely means the thread
> had exited before the process actually is started. Try replacing the
> sleep(2) in the work code with something like sleep(30) -- I
> hypothesize that you'll get the same error condition even with the
> sleep(1) in place.
>
>

Thanks for the reply.

You're right, the error also happens with a longer sleep, or no sleep
at all, inside the function. That sleep is only there in the example to
keep the windows open long enough to see the images if they are
successfully displayed.

I could well be wrong as I'm not fully across multiprocessing, but I
think the Stackoverflow question and answer you linked above relate to a
different situation, with multithreaded cv2 operations on shared image
data.

In my example, AFAIK (which is not very far) it shouldn't matter whether
the new process is sharing the thread, or whether the thread has
exited, because the thread and the process aren't using the same data.
Or (as is quite likely) am I misunderstanding your point?

Cheers

John

[Barry Scott]

> On 29 Aug 2020, at 18:01, Dennis Lee Bieber <wlf...@ix.netcom.com> wrote:
>
> On Sat, 29 Aug 2020 18:24:10 +1000, John O'Hagan <rese...@johnohagan.com>
> declaimed the following:
>
>> There's no error without the sleep(1), nor if the Process is started
>> before the Thread, nor if two Processes are used instead, nor if two
>> Threads are used instead. IOW the error only occurs if a Thread is
>> started first, and a Process is started a little later.
>>
>> Any ideas what might be causing the error?
>>
>
> Under Linux, multiprocessing creates processes using fork(). That means
> that, for some fraction of time, you have TWO processes sharing the same
> thread and all that entails (if it doesn't overlay the forked process with
> a new executable, they are sharing the thread until the thread exits).

In the parent you have 1 or more threads.

After fork the new process has 1 thread, which is not shared with the parent.
Any extra threads are not in the new process. But the memory in the new
process will have data structures from the parents other threads.

So no you never have two processes sharing an threads.

This leads to problems with locks.

Barry

>
> https://stackoverflow.com/questions/54466572/how-to-properly-multithread-in-opencv-in-2019
> (which points to)
> https://answers.opencv.org/question/32415/thread-safe/?answer=32452#post-id-32452
> """
> The library itself is thread safe in that you can have multiple calls into
> the library at the same time, however the data is not always thread safe.
> """
>
> The sleep(1), when compounded with the overhead of starting the thread,
> and then starting the process, likely means the thread had exited before
> the process actually is started. Try replacing the sleep(2) in the work
> code with something like sleep(30) -- I hypothesize that you'll get the
> same error condition even with the sleep(1) in place.
>
>

> --
> Wulfraed Dennis Lee Bieber AF6VN
> wlf...@ix.netcom.com http://wlfraed.microdiversity.freeddns.org/
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

[Stephane Tougard]

On 2020-08-30, Chris Angelico <ros...@gmail.com> wrote:
>> I'm not even that makes sense, how 2 processes can share a thread ?
>>
> They can't. However, they can share a Thread object, which is the
> Python representation of a thread. That can lead to confusion, and
> possibly the OP's error (I don't know for sure, I'm just positing).

A fork() is a copy of a process in a new process. If this process has a
thread (or several), they are part of the copy and the new process has
those threads as well.

Unless there is a memory sharing between those processes, what happens
on one thread in the first process is totally independant of what
happens in the copy of this thread in the other process.

I'm not specialist on multi-threading in Python, but it should not
change anything. Both processes (father and child) don't share the same
thread, each one has its own copy of the thread.

[Barry]

> On 30 Aug 2020, at 11:03, Stephane Tougard via Python-list <pytho...@python.org> wrote:

>
> On 2020-08-30, Chris Angelico <ros...@gmail.com> wrote:
>>> I'm not even that makes sense, how 2 processes can share a thread ?
>>>
>> They can't. However, they can share a Thread object, which is the
>> Python representation of a thread. That can lead to confusion, and
>> possibly the OP's error (I don't know for sure, I'm just positing).
>
> A fork() is a copy of a process in a new process. If this process has a
> thread (or several), they are part of the copy and the new process has
> those threads as well.

No. See https://www.man7.org/linux/man-pages/man2/fork.2.html which says:

“ Note the following further points:

* The child process is created with a single thread—the one that
called fork(). The entire virtual address space of the parent is
replicated in the child, including the states of mutexes,
condition variables, and other pthreads objects; the use of
pthread_atfork(3) may be helpful for dealing with problems that
this can cause.”

Barry

>
> Unless there is a memory sharing between those processes, what happens
> on one thread in the first process is totally independant of what
> happens in the copy of this thread in the other process.
>
> I'm not specialist on multi-threading in Python, but it should not
> change anything. Both processes (father and child) don't share the same
> thread, each one has its own copy of the thread.
>

> --
> https://mail.python.org/mailman/listinfo/python-list
>

[Stephane Tougard]

On 2020-08-30, Barry <ba...@barrys-emacs.org> wrote:
> * The child process is created with a single thread—the one that
> called fork(). The entire virtual address space of the parent is
> replicated in the child, including the states of mutexes,
> condition variables, and other pthreads objects; the use of
> pthread_atfork(3) may be helpful for dealing with problems that

Indeed, I have a similar entry on my NetBSD:

In case of a threaded program, only the thread calling fork() is
still running in the child processes.

Very interesting.

[John O'Hagan]

Thanks to all who replied. I'm still none the wiser about the error, but
I learned something about multiprocessing!

John

[Python]

It's hard to say EXACTLY what the nature of the error is, without
knowing what the underlying libraries are doing. But from this one
thing is clear: You are starting a new thread, and then forking a new
process. This is very likely to break, because contrary to what
several people have said in this thread, the new thread WILL NOT be
copied to the new program, so in the new process, part of your program
is literally missing. This can have all kinds of consequences.

For that reason, you can't spawn a new thread and then fork a new
process, in that order, and expect your program to function correctly.
Whatever your program does that depends on that new thread won't work
correctly, because that thread does not exist in the child (new
process). You can, however, do those things in the reverse order and
it should be fine... new threads will be started in both processes
(unless you take steps to ensure only one of the processes creates the
thread). Both processes retain their integrity and should run fine.

> There's no error without the sleep(1), nor if the Process is started
> before the Thread, nor if two Processes are used instead, nor if two
> Threads are used instead. IOW the error only occurs if a Thread is
> started first, and a Process is started a little later.

Hopefully my explanation above makes it clear why all of those things
are true, other than the sleep() issue. That one is most likely just
a timing issue: Whatever resource is causing the problem hasn't been
set up yet or the critical thread or process has already finished
execution before the issue can arise, or something of the sort. When
you start new threads or processes, usually there's some delay as your
OS schedules each process/thread to run, which is somewhat random
based on how the scheduler works and how loaded the system is. Such
timing problems (bugs that seem to happen randomly with each run of
the program, or over time in a long-executing program) are common in
multi-threaded programs that are written incorrectly, especially when
the state of one thread depends on the state of the other thread, and
the two don't synchronize correctly. This is another way that the
first problem above can manifest, too: The threads can't synchronize
because one of them does not exist!

Hope that helps.