POSIX on Apple

[Peter van Hoof]

Hi,

I had a first stab today at determining the number of available cores on
a Mac. The methods for this are outlined here:

http://stackoverflow.com/questions/150355/programmatically-find-the-number-of-cores-on-a-machine

There are two methods for Mac, either using sysconf() or sysctl(). With
the current default settings on the trunk both methods fail. The
sysconf() method fails because _SC_NPROCESSORS_ONLN is not defined,
while the second methods drowns in compilation errors:

cirrus:source peter$ g++ -W -Wall -c cpu.cpp
In file included from /usr/include/sys/sysctl.h:83,
from cpu.cpp:36:
/usr/include/sys/ucred.h:91: error: 'u_long' does not name a type
/usr/include/sys/ucred.h:133: error: 'u_int' does not name a type
In file included from /usr/include/sys/sysctl.h:84,
from cpu.cpp:36:
/usr/include/sys/proc.h:111: error: 'u_int' does not name a type
/usr/include/sys/proc.h:116: error: 'u_int' does not name a type
/usr/include/sys/proc.h:117: error: 'u_int' does not name a type
/usr/include/sys/proc.h:131: error: 'u_char' does not name a type
/usr/include/sys/proc.h:132: error: 'u_char' does not name a type
/usr/include/sys/proc.h:137: error: 'u_short' does not name a type
/usr/include/sys/proc.h:138: error: 'u_short' does not name a type
In file included from cpu.cpp:36:
/usr/include/sys/sysctl.h:779: error: 'u_int' has not been declared
... etc...

However, if I remove the "#define _POSIX_SOURCE" in cdstd.h both methods
work perfectly fine and detect the number of cores correctly.

Either POSIX is not properly supported on Mac, or we are doing something
wrong here. How should we proceed here?


Cheers,

Peter.

--
Peter van Hoof
Royal Observatory of Belgium
Ringlaan 3
1180 Brussel
Belgium
http://homepage.oma.be/pvh

[Gary J. Ferland]

this says os x is POSIX compliant …

http://stackoverflow.com/questions/5785516/is-osx-a-posix-os 

On Oct 10, 2012, at 5:15 AM, Peter van Hoof <p...@nublado.org> wrote:

POSIX

Gary J. Ferland
Physics, Univ of Kentucky
Lexington KY 40506 USA
tel: 859 257-8795
www.nublado.org

[Robin Williams]

Looks like this may be a bug in FreeBSD ;-(

http://lists.freebsd.org/pipermail/freebsd-bugs/2011-April/044049.html

It's possible that by using u_int, we're assuming a higher variant --
_POSIX_SOURCE should validate the minimum (and working to it should
ensure maximal portability), but there won't be many machines still
functional which only support this level -- apart from perhaps MS,
who've never had much truck with the interface.

Robin

> --
> http://groups.google.com/group/cloudy-dev

[Robin Williams]

...and the Linux man page says

These values also exist, but may not be standard.

- _SC_PHYS_PAGES
The number of pages of physical memory. Note that it is
possible for the product of this value and the value of _SC_PAGE_SIZE
to overflow.

- _SC_AVPHYS_PAGES
The number of currently available pages of physical memory.

- _SC_NPROCESSORS_CONF
The number of processors configured.

- _SC_NPROCESSORS_ONLN
The number of processors currently online (available).

-- so it's possible declaring we're standard-conforming does what it
says in this case, and outlaws this extension. Obviously that would
be good for portability, but...

Robin

[Peter van Hoof]

Hi Robin,

> Looks like this may be a bug in FreeBSD ;-(
>
> http://lists.freebsd.org/pipermail/freebsd-bugs/2011-April/044049.html
>
> It's possible that by using u_int, we're assuming a higher variant --
> _POSIX_SOURCE should validate the minimum (and working to it should
> ensure maximal portability), but there won't be many machines still
> functional which only support this level -- apart from perhaps MS,
> who've never had much truck with the interface.

That report does indeed look very similar. Note that the missing types are used
in header files, so there is nothing we can do about that. So I guess the
proposed workaround would be to change cdstd.h to

// defining _POSIX_SOURCE on Darwin will cause compilation errors about missing
// type definitions (e.g., u_int) when including certain system header files
#ifndef __APPLE__
#define _POSIX_SOURCE
#endif

or do you see a better solution? We should also include FreeBSD (and maybe other
BSD variants??), but I doubt we have any actual users on those.


Cheers,

Peter.

> On 10 October 2012 12:51, Gary J. Ferland <gjfe...@gmail.com> wrote:

>> this says os x is POSIX compliant �

>> http://stackoverflow.com/questions/5785516/is-osx-a-posix-os
>>
>> On Oct 10, 2012, at 5:15 AM, Peter van Hoof <p...@nublado.org> wrote:
>>
>> POSIX
>>
>>
>>
>> Gary J. Ferland
>> Physics, Univ of Kentucky
>> Lexington KY 40506 USA
>> tel: 859 257-8795
>> www.nublado.org
>>
>> --
>> http://groups.google.com/group/cloudy-dev
>

[Peter van Hoof]

On 2012-10-10 20:31, Robin Williams wrote:
> ...and the Linux man page says
>
> These values also exist, but may not be standard.
>
> - _SC_PHYS_PAGES
> The number of pages of physical memory. Note that it is
> possible for the product of this value and the value of _SC_PAGE_SIZE
> to overflow.
>
> - _SC_AVPHYS_PAGES
> The number of currently available pages of physical memory.
>
> - _SC_NPROCESSORS_CONF
> The number of processors configured.
>
> - _SC_NPROCESSORS_ONLN
> The number of processors currently online (available).
>
> -- so it's possible declaring we're standard-conforming does what it
> says in this case, and outlaws this extension. Obviously that would
> be good for portability, but...

_SC_NPROCESSORS_ONLN is certainly not standardized (see the code in cpu.cpp). It
looks like Linux does not undefine this in the current setup though. I assume
that the sysctl() method listed on the stackoverflow.com page is the official
POSIX way of doing things?

Peter.

> On 10 October 2012 18:59, Robin Williams <rjrwi...@googlemail.com> wrote:
>> Looks like this may be a bug in FreeBSD ;-(
>>
>> http://lists.freebsd.org/pipermail/freebsd-bugs/2011-April/044049.html
>>
>> It's possible that by using u_int, we're assuming a higher variant --
>> _POSIX_SOURCE should validate the minimum (and working to it should
>> ensure maximal portability), but there won't be many machines still
>> functional which only support this level -- apart from perhaps MS,
>> who've never had much truck with the interface.
>>
>> Robin
>>
>> On 10 October 2012 12:51, Gary J. Ferland <gjfe...@gmail.com> wrote:

>>> this says os x is POSIX compliant �

>>> http://stackoverflow.com/questions/5785516/is-osx-a-posix-os
>>>
>>> On Oct 10, 2012, at 5:15 AM, Peter van Hoof <p...@nublado.org> wrote:
>>>
>>> POSIX
>>>
>>>
>>>
>>> Gary J. Ferland
>>> Physics, Univ of Kentucky
>>> Lexington KY 40506 USA
>>> tel: 859 257-8795
>>> www.nublado.org
>>>
>>> --
>>> http://groups.google.com/group/cloudy-dev
>

[Robin Williams]

Yes, that seems like the best way through this.

It looks like the sysconf variant is more likely to make it into a
standard /someday/, so personally I'd go with that, though it rules
out old MacOS implementations. I guess old versions are less likely
to be on multicore hardware, so perhaps there's not much loss here.

Robin

On 10 October 2012 19:49, Peter van Hoof <p...@nublado.org> wrote:
> Hi Robin,
>
>
>> Looks like this may be a bug in FreeBSD ;-(
>>
>> http://lists.freebsd.org/pipermail/freebsd-bugs/2011-April/044049.html
>>
>> It's possible that by using u_int, we're assuming a higher variant --
>> _POSIX_SOURCE should validate the minimum (and working to it should
>> ensure maximal portability), but there won't be many machines still
>> functional which only support this level -- apart from perhaps MS,
>> who've never had much truck with the interface.
>
>
> That report does indeed look very similar. Note that the missing types are
> used in header files, so there is nothing we can do about that. So I guess
> the proposed workaround would be to change cdstd.h to
>
> // defining _POSIX_SOURCE on Darwin will cause compilation errors about
> missing
> // type definitions (e.g., u_int) when including certain system header files
> #ifndef __APPLE__
> #define _POSIX_SOURCE
> #endif
>
> or do you see a better solution? We should also include FreeBSD (and maybe
> other BSD variants??), but I doubt we have any actual users on those.
>
>
> Cheers,
>
> Peter.
>
>
>> On 10 October 2012 12:51, Gary J. Ferland <gjfe...@gmail.com> wrote:
>>>

>>> this says os x is POSIX compliant …

>>> http://stackoverflow.com/questions/5785516/is-osx-a-posix-os
>>>
>>> On Oct 10, 2012, at 5:15 AM, Peter van Hoof <p...@nublado.org> wrote:
>>>
>>> POSIX
>>>
>>>
>>>
>>> Gary J. Ferland
>>> Physics, Univ of Kentucky
>>> Lexington KY 40506 USA
>>> tel: 859 257-8795
>>> www.nublado.org
>>>
>>> --
>>> http://groups.google.com/group/cloudy-dev
>>
>>
>
>
> --
> Peter van Hoof
> Royal Observatory of Belgium
> Ringlaan 3
> 1180 Brussel
> Belgium
> http://homepage.oma.be/pvh
>

> --
> http://groups.google.com/group/cloudy-dev

[Robin Williams]

http://linux.die.net/man/2/sysctl

sysctl(2) - Linux man page
Name

sysctl - read/write system parameters
Synopsis

#include <unistd.h>
#include <linux/sysctl.h>

int _sysctl(struct __sysctl_args *args);

Description

Do not use this system call! See NOTES.

>>>> this says os x is POSIX compliant …

>>>> http://stackoverflow.com/questions/5785516/is-osx-a-posix-os
>>>>
>>>> On Oct 10, 2012, at 5:15 AM, Peter van Hoof <p...@nublado.org> wrote:
>>>>
>>>> POSIX
>>>>
>>>>
>>>>
>>>> Gary J. Ferland
>>>> Physics, Univ of Kentucky
>>>> Lexington KY 40506 USA
>>>> tel: 859 257-8795
>>>> www.nublado.org
>>>>
>>>> --
>>>> http://groups.google.com/group/cloudy-dev
>>
>>
>
>
> --
> Peter van Hoof
> Royal Observatory of Belgium
> Ringlaan 3
> 1180 Brussel
> Belgium
> http://homepage.oma.be/pvh
>

> --
> http://groups.google.com/group/cloudy-dev

[Peter van Hoof]

Hi Robin,

> Yes, that seems like the best way through this.
>
> It looks like the sysconf variant is more likely to make it into a
> standard /someday/, so personally I'd go with that, though it rules
> out old MacOS implementations. I guess old versions are less likely
> to be on multicore hardware, so perhaps there's not much loss here.

OK, so I guess things will stay messy... That is OK, it is what cpu.cpp
is there for: to hide this kind of system mess behind a uniform
interface for the rest of the code.

Things are now implemented as

#if defined(some_symbol)
#elif defined(some_other_symbol)
... etc ...
#endif

This works quite well as the symbols used in the various methods have
quite rich names (and are all preprocessor defined), so there is very
little chance of them accidentally meaning something else on another
system. I will use that for the Apple methods as well, so picking the
correct method should be transparent. I will keep the sysconf method and
add the sysctl method for Apple/BSD only. Together with the fix below, I
think that should address all our concerns.

>>>> this says os x is POSIX compliant �