Debian and Ubuntu package

[Daniel Pocock]

There is now an official Debian package of Concurrency Kit, also available on Ubuntu

    http://packages.qa.debian.org/c/ck.html

However, the builds are failing on several platforms and running indefinitely on i386:

    https://buildd.debian.org/status/package.php?p=ck&suite=experimental

The build flags, etc, are visible in the build logs.  The build script for Debian is here:

http://anonscm.debian.org/gitweb/?p=collab-maint/ck.git;a=blob;f=debian/rules;hb=HEAD

Can anybody comment on any of these issues and correct them for the next release?

[Samy Al Bahra]

Sure. I will take a look. On i386, what is the timeout? Tests can take a long time to run.

--
You received this message because you are subscribed to the Google Groups "Concurrency Kit" group.
To unsubscribe from this group and stop receiving emails from it, send an email to concurrencyki...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Daniel Pocock]

On 09/04/14 15:05, Samy Al Bahra wrote:

Sure. I will take a look. On i386, what is the timeout? Tests can take a long time to run.

The tests ran in 7 minutes on amd64 before completing successfully

The i386 buildd shows it has been trying for 1 hour before it failed:

https://buildd.debian.org/status/fetch.php?pkg=ck&arch=i386&ver=0.4.1-1&stamp=1397045023

[João Fernandes]

I just tested on a i386 machine (actually a VM) and the barrier_tournament test doesn't seem to terminate...

[Samy Al Bahra]

How long was it running for Joao?

Samy Al Bahra [http://repnop.org]

[Samy Al Bahra]

How many threads? How many cores? Some stack trace information would be useful.

[Daniel Pocock]

On 09/04/14 15:18, Samy Al Bahra wrote:
> How many threads? How many cores? Some stack trace information would be useful.

Samy, our Debian buildd servers do automated builds on different
machines at different times

All the log output is available through the web interface

If there are details you may need about the build environment when
troubleshooting, could you add something to the Makefile to output the
necessary details before starting the build? Then everybody (including
you) will be able to look at the console output on buildd.debian.org

[João Fernandes]

Not much time, around 10 minutes. Single core machine. Let me try to get more info on this.

[João Fernandes]

Interestingly, from the logs pointed out by Daniel it seems that it in their case was the bytelock test that didn't terminate.

[Samy Al Bahra]

Sorry Daniel, that was intended for Joao :-) The necessary information
is already available in the build output. I will investigate the
issues on i386 with ck_bytelock and get back to you.

[Samy Al Bahra]

Hi Joao,

The tournament barrier assumes more than 1 thread, this isn't
i386-specific. I will fix this.

[João Fernandes]

I see, sorry for adding noise to the thread!

[Samy Al Bahra]

ck_barrier_tournament now supports the unlikely case where it is only
to be used for synchronizing up to one thread. It's in master and will
be included next release.

[Daniel Pocock]

On 09/04/14 16:13, Samy Al Bahra wrote:
> ck_barrier_tournament now supports the unlikely case where it is only
> to be used for synchronizing up to one thread. It's in master and will
> be included next release.

Can you also have a look at the build failures on other platforms, for
example:

https://bugs.debian.org/cgi-bin/pkgreport.cgi?ordering=normal;archive=0;src=ck;dist=unstable;repeatmerged=0


The build log for ARM:

https://buildd.debian.org/status/fetch.php?pkg=ck&arch=armel&ver=0.4.1-1&stamp=1397035976


Let me know when you make another release and I will get the package
updated.

[Samy Al Bahra]

The ck_bytelock issue has also been fixed in master, will be included
next release.

On Wed, Apr 9, 2014 at 10:36 AM, Daniel Pocock <dan...@pocock.pro> wrote:
> On 09/04/14 16:13, Samy Al Bahra wrote:
>> ck_barrier_tournament now supports the unlikely case where it is only
>> to be used for synchronizing up to one thread. It's in master and will
>> be included next release.
>
>
> Can you also have a look at the build failures on other platforms, for
> example:

Yes, these are primarily build system related for SPARC and Power.
I'll look into this.

MIPS, armel and s390x are simply unsupported, however (armhf is
another story, but I am waiting for the run there). How can we
indicate to Debian that CK should not be built for a certain set of
architectures (x86, x86-64, sparcv9+, ppc32, ppc64, armv6+).

>
> https://bugs.debian.org/cgi-bin/pkgreport.cgi?ordering=normal;archive=0;src=ck;dist=unstable;repeatmerged=0
>
>
> The build log for ARM:
>
> https://buildd.debian.org/status/fetch.php?pkg=ck&arch=armel&ver=0.4.1-1&stamp=1397035976
>
>
> Let me know when you make another release and I will get the package
> updated.
>
>

[Daniel Pocock]

On 09/04/14 16:50, Samy Al Bahra wrote:
> The ck_bytelock issue has also been fixed in master, will be included
> next release.
>
> On Wed, Apr 9, 2014 at 10:36 AM, Daniel Pocock <dan...@pocock.pro> wrote:
>> On 09/04/14 16:13, Samy Al Bahra wrote:
>>> ck_barrier_tournament now supports the unlikely case where it is only
>>> to be used for synchronizing up to one thread. It's in master and will
>>> be included next release.
>>
>> Can you also have a look at the build failures on other platforms, for
>> example:
> Yes, these are primarily build system related for SPARC and Power.
> I'll look into this.
>
> MIPS, armel and s390x are simply unsupported, however (armhf is
> another story, but I am waiting for the run there). How can we
> indicate to Debian that CK should not be built for a certain set of
> architectures (x86, x86-64, sparcv9+, ppc32, ppc64, armv6+).

I can put a restriction in the debian/control file to indicate the
package is only aimed at supported platforms.

However, ARM is very popular now. People who make other packages won't
use your library at all if they think it won't work on ARM.

On the other hand, the community is very supportive - you may find some
of the ARM developers on the mailing lists willing to offer
suggestions. Debian also has some special lab servers sponsored by ARM
where you can log in and test your code, I can try and request access
for you. There is a dedicated mailing list for ARM here:

https://lists.debian.org/debian-arm/

[Samy Al Bahra]

Hi Daniel,

ARMv6+ is supported, and has been supported for a while. The problem
here is that armel doesn't really support any interesting atomic
operations. We would be putting a restriction on armel, not ARM at
large.

[Daniel Pocock]

On 09/04/14 17:16, Samy Al Bahra wrote:
> Hi Daniel,
>
> ARMv6+ is supported, and has been supported for a while. The problem
> here is that armel doesn't really support any interesting atomic
> operations. We would be putting a restriction on armel, not ARM at
> large.

People using Concurrency Kit in their code may sometimes want to build
their code on platforms that have only a single CPU/core

In such single-CPU/core hardware, the code may run successfully without
the need for all the atomic primitives that are necessary in
multi-CPU/multi-core platforms. armel is in this category - there is no
multicore armel as far as I know.

Is there any way Concurrency Kit can support these platforms? E.g. so
that calls to the ck API functions will appear to operate normally and
the user will not need to put lots of #ifdef logic in their own code?

The project I'm working with is Ganglia - it is also open source.
Previous versions did not use Concurrency Kit and we were always able to
compile it on all the Debian architectures, it would be nice to have an
efficient way to continue supporting those architectures.

[Samy Al Bahra]

On Fri, May 2, 2014 at 9:33 AM, Daniel Pocock <dan...@pocock.pro> wrote:
[...]

>
>
> People using Concurrency Kit in their code may sometimes want to build
> their code on platforms that have only a single CPU/core

CK supports such environments, no problem. They just need to actually
have a universal atomic primitive.

>
> In such single-CPU/core hardware, the code may run successfully without
> the need for all the atomic primitives that are necessary in
> multi-CPU/multi-core platforms. armel is in this category - there is no
> multicore armel as far as I know.

In such environments, you would need locks with user-defined yield
functionality to serialize atomic operations or restartable critical
sections (which implies a lot of MD-specific code). My opinion is that
these will be more trouble than they're worth. We currently require
platforms to at least have a single universal atomic primitive.

>
> Is there any way Concurrency Kit can support these platforms? E.g. so
> that calls to the ck API functions will appear to operate normally and
> the user will not need to put lots of #ifdef logic in their own code?

Olivier is the maintainer of the ARM port. Olivier, what are your
thoughts on these older ARM architectures?

>
> The project I'm working with is Ganglia - it is also open source.
> Previous versions did not use Concurrency Kit and we were always able to
> compile it on all the Debian architectures, it would be nice to have an
> efficient way to continue supporting those architectures.
>

[Olivier Houchard]

Hi,

 

On Friday, May 2, 2014 5:15:17 PM UTC+2, sbahra wrote:

>
> In such single-CPU/core hardware, the code may run successfully without
> the need for all the atomic primitives that are necessary in
> multi-CPU/multi-core platforms.  armel is in this category - there is no
> multicore armel as far as I know.

In such environments, you would need locks with user-defined yield
functionality to serialize atomic operations or restartable critical
sections (which implies a lot of MD-specific code). My opinion is that
these will be more trouble than they're worth. We currently require
platforms to at least have a single universal atomic primitive. 

>
> Is there any way Concurrency Kit can support these platforms?  E.g. so
> that calls to the ck API functions will appear to operate normally and
> the user will not need to put lots of #ifdef logic in their own code?

Olivier is the maintainer of the ARM port. Olivier, what are your
thoughts on these older ARM architectures?

 That is true, arm <= v6 doesn't provide any useful atomic instructions do

do implement CAS or FAS. Even if you're not running with multiple cores, 

you can still be preempted, and if it happens while in the middle of a CAS,

it might be not so good. Linux provides a mechanism to do restartable

critical sections, so it may be done, but I'm not sure it's worth it either.

That would require us to add lots of #ifdef in CK :)

I think armv5 is mostly dead, more and more Linux distributions are dropping

support for it, and I'm not sure there's a point in working on this.

However, if it is felt that it is required, I may be convinced to do it.

Regards,

Olivier

--
Samy Al Bahra [http://repnop.org]