Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
A book for learning threads?
129 views
Skip to first unread message
Robert Miles
Oct 28, 2012, 8:26:29 PM10/28/12
to
I'm updating a program using threads so that it can use checkpoints instead
of often running for a month or more without the ability to restart anywhere
except at the beginning.
It will later be recompiled for multiple platforms, but for now I am
only using
the Cygwin version of gcc under 64-bit Windows. Without checkpoints, it
will
already compile for all the intended platforms, using other versions of gcc.
It uses the Posix threads method of getting threads (pthreads). However,
I've
never used threads before. Also, I've never used Linux, and have done
little
under Unix.
I've already found that my K&R 2 book does not mention pthreads in the
index.
What book would you recommend for learning enough about threads and
pthreads that I can do a reasonable job with this update? I'd also
consider an
online class, but not an on-campus class.
of often running for a month or more without the ability to restart anywhere
except at the beginning.
It will later be recompiled for multiple platforms, but for now I am
only using
the Cygwin version of gcc under 64-bit Windows. Without checkpoints, it
will
already compile for all the intended platforms, using other versions of gcc.
It uses the Posix threads method of getting threads (pthreads). However,
I've
never used threads before. Also, I've never used Linux, and have done
little
under Unix.
I've already found that my K&R 2 book does not mention pthreads in the
index.
What book would you recommend for learning enough about threads and
pthreads that I can do a reasonable job with this update? I'd also
consider an
online class, but not an on-campus class.
Lucas Levrel
Oct 29, 2012, 4:45:14 AM10/29/12
to
Le 28 octobre 2012, Robert Miles a écrit :
> What book would you recommend for learning enough about threads and
> pthreads that I can do a reasonable job with this update? I'd also
> consider an online class, but not an on-campus class.
You should specify what kind of app you're programming. There are plenty
> What book would you recommend for learning enough about threads and
> pthreads that I can do a reasonable job with this update? I'd also
> consider an online class, but not an on-campus class.
of books, many are specialized.
--
LL
Mark
Oct 29, 2012, 5:45:39 AM10/29/12
to
(2nd Edition)".
<http://www.amazon.com/Advanced-Programming-UNIX-Environment-Edition/dp/0201433079>
I only have the first edition which does not include threading but the
second edition has.
--
(\__/) M.
(='.'=) If a man stands in a forest and no woman is around
(")_(") is he still wrong?
Rui Maciel
Oct 29, 2012, 8:52:54 AM10/29/12
to
Robert Miles wrote:
> I'm updating a program using threads so that it can use checkpoints
> instead of often running for a month or more without the ability to
> restart anywhere except at the beginning.
>
> It will later be recompiled for multiple platforms, but for now I am
> only using
> the Cygwin version of gcc under 64-bit Windows. Without checkpoints, it
> will
> already compile for all the intended platforms, using other versions of
> gcc.
>
> It uses the Posix threads method of getting threads (pthreads). However,
> I've
> never used threads before. Also, I've never used Linux, and have done
> little
> under Unix.
>
> I've already found that my K&R 2 book does not mention pthreads in the
> index.
Yes. As far as I know, the last edition of K&R dates back to 1988, and the
> I'm updating a program using threads so that it can use checkpoints
> instead of often running for a month or more without the ability to
> restart anywhere except at the beginning.
>
> It will later be recompiled for multiple platforms, but for now I am
> only using
> the Cygwin version of gcc under 64-bit Windows. Without checkpoints, it
> will
> already compile for all the intended platforms, using other versions of
> gcc.
>
> It uses the Posix threads method of getting threads (pthreads). However,
> I've
> never used threads before. Also, I've never used Linux, and have done
> little
> under Unix.
>
> I've already found that my K&R 2 book does not mention pthreads in the
> index.
POSIX threads standard was only published in 1995.
> What book would you recommend for learning enough about threads and
> pthreads that I can do a reasonable job with this update?
pthreads programming book a good read to get up to speed on pthreads.
Hope this helps,
Rui Maciel
Robert Miles
Oct 29, 2012, 10:42:18 AM10/29/12
to
It's related to RNA research and runs under BOINC. It needs to be able to
run on multiple types of platforms, including some with Windows and some with Linux.
It uses the easel and hmmer libraries as part of the source code.
Robert Miles
Oct 29, 2012, 10:45:34 AM10/29/12
to rui.m...@gmail.com
On Monday, October 29, 2012 7:52:55 AM UTC-5, Rui Maciel wrote:
> Robert Miles wrote:
[snip]
Robert Miles
> Robert Miles wrote:
[snip]
> > What book would you recommend for learning enough about threads and
> > pthreads that I can do a reasonable job with this update?
>
> I doubt it is the best book written on the subject, but I found O'Reilly's
> pthreads programming book a good read to get up to speed on pthreads.
>
> Hope this helps,
>
> Rui Maciel
Thank you.
> > pthreads that I can do a reasonable job with this update?
>
> I doubt it is the best book written on the subject, but I found O'Reilly's
> pthreads programming book a good read to get up to speed on pthreads.
>
> Hope this helps,
>
> Rui Maciel
Robert Miles
Lucas Levrel
Oct 29, 2012, 12:35:13 PM10/29/12
to
Le 29 octobre 2012, Robert Miles a ï¿œcrit :
> On Monday, October 29, 2012 3:45:15 AM UTC-5, Lucas Levrel wrote:
>> Le 28 octobre 2012, Robert Miles a ï¿œcrit :
(in my opinion). If you need a primer on parallel programming, there are:
- The Art of Concurrency: introduces a classification of parallel
algorithms, which may help you identifying what threads are used for;
- An Introduction to Parallel Programming: also deals with other forms of
parallelism (MPI), but is science-oriented.
> It needs to be able to run on multiple types of platforms, including
> some with Windows and some with Linux.
If it already uses Pthreads and compiles on these platforms, where's the
worry? Just stick with the Pthread implementation of the threads concept.
--
LL
> On Monday, October 29, 2012 3:45:15 AM UTC-5, Lucas Levrel wrote:
>>
>>> What book would you recommend for learning enough about threads and
>>> pthreads that I can do a reasonable job with this update? I'd also
>>> consider an online class, but not an on-campus class.
>>
>> You should specify what kind of app you're programming. There are plenty
>> of books, many are specialized.
>
> I'm updating an existing program rather than writing a new program.
>
> It's related to RNA research and runs under BOINC.
Then, be aware that many books are more IT-oriented than science-oriented
>>> What book would you recommend for learning enough about threads and
>>> pthreads that I can do a reasonable job with this update? I'd also
>>> consider an online class, but not an on-campus class.
>>
>> You should specify what kind of app you're programming. There are plenty
>> of books, many are specialized.
>
> I'm updating an existing program rather than writing a new program.
>
> It's related to RNA research and runs under BOINC.
(in my opinion). If you need a primer on parallel programming, there are:
- The Art of Concurrency: introduces a classification of parallel
algorithms, which may help you identifying what threads are used for;
- An Introduction to Parallel Programming: also deals with other forms of
parallelism (MPI), but is science-oriented.
> It needs to be able to run on multiple types of platforms, including
> some with Windows and some with Linux.
worry? Just stick with the Pthread implementation of the threads concept.
--
LL
Robert Miles
Oct 30, 2012, 2:49:37 AM10/30/12
to
On Monday, October 29, 2012 11:35:14 AM UTC-5, Lucas Levrel wrote:
BOINC projects generally have budgets low enough that they need to ask volunteers for computer time on some of the computers less reliable at running for a month or more without any interruption, such as computers running Windows and without any UPS to help them through power flickers. Therefore, it is a very useful feature to have checkpoints that can frequently save the work in progress to a checkpoint, and resume from that checkpoint later if there is some interruption such as a power outage for a few seconds longer than the computer can run without power.
I don't expect any changes to be needed for the main purpose of the program, only the very many places I need to change for adding a checkpoint feature.
> Le 29 octobre 2012, Robert Miles a écrit :
>
>
>
> > On Monday, October 29, 2012 3:45:15 AM UTC-5, Lucas Levrel wrote:
>
>
>
>
> > On Monday, October 29, 2012 3:45:15 AM UTC-5, Lucas Levrel wrote:
>
> >> Le 28 octobre 2012, Robert Miles a écrit :
>
> >>
>
> >>> What book would you recommend for learning enough about threads and
>
> >>> pthreads that I can do a reasonable job with this update? I'd also
>
> >>> consider an online class, but not an on-campus class.
>
> >>
>
> >> You should specify what kind of app you're programming. There are plenty
>
> >> of books, many are specialized.
>
> >
>
> > I'm updating an existing program rather than writing a new program.
>
> >
>
> > It's related to RNA research and runs under BOINC.
>
>
>
> Then, be aware that many books are more IT-oriented than science-oriented
>
> (in my opinion). If you need a primer on parallel programming, there are:
>
> - The Art of Concurrency: introduces a classification of parallel
>
> algorithms, which may help you identifying what threads are used for;
>
> - An Introduction to Parallel Programming: also deals with other forms of
>
> parallelism (MPI), but is science-oriented.
>
>
>
> > It needs to be able to run on multiple types of platforms, including
>
> > some with Windows and some with Linux.
>
>
>
> If it already uses Pthreads and compiles on these platforms, where's the
>
> worry? Just stick with the Pthread implementation of the threads concept.
There's no desire to have it run only on platforms that can run for a month or more without any interruptions. Adding checkpoints so that it can resume with less computer time lost due to any interruptions REQUIRES understanding threads so that I can suspend all threads except the main thread when writing checkpoints or restoring from them. I don't yet know if it also requires telling all the other threads to place information specific to those threads where the main thread can find it and include it in the checkpoint.
>
> >>
>
> >>> What book would you recommend for learning enough about threads and
>
> >>> pthreads that I can do a reasonable job with this update? I'd also
>
> >>> consider an online class, but not an on-campus class.
>
> >>
>
> >> You should specify what kind of app you're programming. There are plenty
>
> >> of books, many are specialized.
>
> >
>
> > I'm updating an existing program rather than writing a new program.
>
> >
>
> > It's related to RNA research and runs under BOINC.
>
>
>
> Then, be aware that many books are more IT-oriented than science-oriented
>
> (in my opinion). If you need a primer on parallel programming, there are:
>
> - The Art of Concurrency: introduces a classification of parallel
>
> algorithms, which may help you identifying what threads are used for;
>
> - An Introduction to Parallel Programming: also deals with other forms of
>
> parallelism (MPI), but is science-oriented.
>
>
>
> > It needs to be able to run on multiple types of platforms, including
>
> > some with Windows and some with Linux.
>
>
>
> If it already uses Pthreads and compiles on these platforms, where's the
>
> worry? Just stick with the Pthread implementation of the threads concept.
BOINC projects generally have budgets low enough that they need to ask volunteers for computer time on some of the computers less reliable at running for a month or more without any interruption, such as computers running Windows and without any UPS to help them through power flickers. Therefore, it is a very useful feature to have checkpoints that can frequently save the work in progress to a checkpoint, and resume from that checkpoint later if there is some interruption such as a power outage for a few seconds longer than the computer can run without power.
I don't expect any changes to be needed for the main purpose of the program, only the very many places I need to change for adding a checkpoint feature.
Lucas Levrel
Oct 30, 2012, 5:43:39 AM10/30/12
to
Le 29 octobre 2012, Robert Miles a écrit :
> On Monday, October 29, 2012 11:35:14 AM UTC-5, Lucas Levrel wrote:
>> Le 29 octobre 2012, Robert Miles a écrit :
>>
>>> It needs to be able to run on multiple types of platforms, including
>>> some with Windows and some with Linux.
>>
>> If it already uses Pthreads and compiles on these platforms, where's the
>> worry? Just stick with the Pthread implementation of the threads concept.
>
> There's no desire to have it run only on platforms that can run for a
> month or more without any interruptions. Adding checkpoints so that it
> can resume with less computer time lost due to any interruptions
> REQUIRES understanding threads so that I can suspend all threads except
> the main thread when writing checkpoints or restoring from them.
Of course. I was responding to "it needs to be able to run on multiple
> On Monday, October 29, 2012 11:35:14 AM UTC-5, Lucas Levrel wrote:
>> Le 29 octobre 2012, Robert Miles a écrit :
>>
>>> It needs to be able to run on multiple types of platforms, including
>>> some with Windows and some with Linux.
>>
>> If it already uses Pthreads and compiles on these platforms, where's the
>> worry? Just stick with the Pthread implementation of the threads concept.
>
> There's no desire to have it run only on platforms that can run for a
> month or more without any interruptions. Adding checkpoints so that it
> can resume with less computer time lost due to any interruptions
> REQUIRES understanding threads so that I can suspend all threads except
> the main thread when writing checkpoints or restoring from them.
types of platforms". Isn't it already?
--
LL
Robert Miles
Oct 30, 2012, 10:27:27 AM10/30/12
to
Markus Elfring
Nov 1, 2012, 8:31:57 AM11/1/12
to
> I don't expect any changes to be needed for the main purpose of the program, only the very many places I need to change for adding a checkpoint feature.
Can you reuse any software library for the needed functionality "checkpointing"?
Regards,
Markus
Rui Maciel
Nov 1, 2012, 1:55:28 PM11/1/12
to
Robert Miles wrote:
> There's no desire to have it run only on platforms that can run for a
> month or more without any interruptions. Adding checkpoints so that it
> can resume with less computer time lost due to any interruptions REQUIRES
> understanding threads so that I can suspend all threads except the main
> thread when writing checkpoints or restoring from them. I don't yet know
> if it also requires telling all the other threads to place information
> specific to those threads where the main thread can find it and include it
> in the checkpoint.
The pthreads standard defines essentially a set of primitive operations,
> There's no desire to have it run only on platforms that can run for a
> month or more without any interruptions. Adding checkpoints so that it
> can resume with less computer time lost due to any interruptions REQUIRES
> understanding threads so that I can suspend all threads except the main
> thread when writing checkpoints or restoring from them. I don't yet know
> if it also requires telling all the other threads to place information
> specific to those threads where the main thread can find it and include it
> in the checkpoint.
which are then put together to pull off whatever concurency tricks you wish
to pull. This means that implementing the "checkpoint" feature you referred
to isn't necessarily a pthreads issue; only a software design one.
You didn't said what you mean by "checkpoint". Without any specific
details, it might be possible that it only take a single mutex to be able to
implement your "checkpoints" feature. Nevertheless, this only depends on
what your requirements are, and how you designed/will design your software.
Pthreads is orthogonal to this.
Robert Miles
Nov 1, 2012, 9:39:30 PM11/1/12
to
It is REQUIRED that no other threads are still changing the state of the program while this is done.
At least under Windows, the memory will probably be rearranged when the program restarts, so just writing everything back to the same addresses in memory will not work - some of its previous memory is now likely to already be in use by some other program.
Ian Collins
Nov 2, 2012, 9:52:53 PM11/2/12
to
On 10/30/12 19:49, Robert Miles wrote:
> On Monday, October 29, 2012 11:35:14 AM UTC-5, Lucas Levrel wrote:
>>
>> If it already uses Pthreads and compiles on these platforms, where's the
>> worry? Just stick with the Pthread implementation of the threads concept.
>
Please trim your lines and tidy up the mess google makes of your replies!
> On Monday, October 29, 2012 11:35:14 AM UTC-5, Lucas Levrel wrote:
>>
>> If it already uses Pthreads and compiles on these platforms, where's the
>> worry? Just stick with the Pthread implementation of the threads concept.
>
> There's no desire to have it run only on platforms that can run for a month or more without any interruptions. Adding checkpoints so that it can resume with less computer time lost due to any interruptions REQUIRES understanding threads so that I can suspend all threads except the main thread when writing checkpoints or restoring from them. I don't yet know if it also requires telling all the other threads to place information specific to those threads where the main thread can find it and include it in the checkpoint..
Applications like BOINC typically dish out units of work to client
machines and these in turn pass them on to worker threads. In this
case, it makes sense for the unit of work to be scaled to be a) useful
and b) small enough so not too much is wasted if the machine dies during
processing.
Saving state part way through a work unit probably isn't worth while.
Make your "checkpoints" the completion of a work unit.
--
Ian Collins
Robert Miles
Nov 3, 2012, 2:46:06 AM11/3/12
to
On Friday, November 2, 2012 8:52:55 PM UTC-5, Ian Collins wrote:
> On 10/30/12 19:49, Robert Miles wrote:
>
> > On Monday, October 29, 2012 11:35:14 AM UTC-5, Lucas Levrel wrote:
>
> >> If it already uses Pthreads and compiles on these platforms, where's the
> >> worry? Just stick with the Pthread implementation of the threads concept.
>
> Please trim your lines and tidy up the mess google makes of your replies!
Trying.
> On 10/30/12 19:49, Robert Miles wrote:
>
> > On Monday, October 29, 2012 11:35:14 AM UTC-5, Lucas Levrel wrote:
>
> >> If it already uses Pthreads and compiles on these platforms, where's the
> >> worry? Just stick with the Pthread implementation of the threads concept.
>
> Please trim your lines and tidy up the mess google makes of your replies!
> > There's no desire to have it run only on platforms that can run for a
> > month or more without any interruptions. Adding checkpoints so that it
> > can resume with less computer time lost due to any interruptions REQUIRES
> > understanding threads so that I can suspend all threads except the main
> > thread when writing checkpoints or restoring from them. I don't yet know
> > if it also requires telling all the other threads to place information
> > specific to those threads where the main thread can find it and include it
> > in the checkpoint..
>
> Applications like BOINC typically dish out units of work to client
> machines and these in turn pass them on to worker threads. In this
> case, it makes sense for the unit of work to be scaled to be a) useful
> and b) small enough so not too much is wasted if the machine dies during
> processing.
>
> Saving state part way through a work unit probably isn't worth while.
>
> Make your "checkpoints" the completion of a work unit.
be less than a month long, and sometimes longer. If there was, checkpoints
would already be available.
There's also a much smaller type of workunits for the much shorter RNAs; these
already have checkpoints at the end of each section of RNA. However, this doesn't work for the larger RNAs, such as most of those found in humans and
other mammals. The nature of the problem just doesn't allow doing much with
arbitrarily chopped pieces of the RNA.
Markus Elfring
Nov 5, 2012, 1:58:21 PM11/5/12
to
> Deciding what to write is very program-dependent;
I agree ...
> I do not expect any software library to do a decent job at deciding this.
http://en.wikipedia.org/wiki/Application_checkpointing#Practical_implementations_for_Linux.2FUnix
http://boinc.berkeley.edu/trac/wiki/BasicApi#checkpointing
Regards,
Markus
Markus Elfring
Nov 5, 2012, 2:00:09 PM11/5/12
to
> Deciding what to write is very program-dependent;
I agree ...
> I do not expect any software library to do a decent job at deciding this.
Ian Collins
Nov 6, 2012, 4:37:48 PM11/6/12
to
one. Surly it would be easier for the work unit to know when would be a
good time to dump its state rather than rely on an application wide
mechanism? If the computations use multiple threads, each thread could
save its own state.
--
Ian Collins
0 new messages