Building on OSX

[Kenton Varda]

I've added instructions for building Cap'n Proto on Mac OSX, without help from macports/fink/whatever.  It requires downloading newer Clang binaries, as the ones that come with Xcode lie about their version and are not actually new enough.

http://kentonv.github.io/capnproto/install.html

The code has been updated to check for problems as well.

Going forward I intend for OSX to be a supported platform via these instructions.  (It ought to work with macports / fink / whatever as well, probably more easily.)

I'm going to take a pass at MinGW support as well...

-Kenton

[Charles Strahan]

Hi Kenton,

I followed the directions for building on OSX, but it seems that Clang 3.2 (and 3.3) fails:

./configure CXX=~/clang-3.3/bin/clang++

checking for a BSD-compatible install... /usr/bin/install -c

checking whether build environment is sane... yes

checking for a thread-safe mkdir -p... build-aux/install-sh -c -d

checking for gawk... no

checking for mawk... no

checking for nawk... no

checking for awk... awk

checking whether make sets $(MAKE)... yes

checking whether make supports nested variables... yes

checking for gcc... gcc

checking whether the C compiler works... yes

checking for C compiler default output file name... a.out

checking for suffix of executables...

checking whether we are cross compiling... no

checking for suffix of object files... o

checking whether we are using the GNU C compiler... yes

checking whether gcc accepts -g... yes

checking for gcc option to accept ISO C89... none needed

checking for style of include used by make... GNU

checking dependency style of gcc... gcc3

checking whether we are using the GNU C++ compiler... no

checking whether ~/clang-3.3/bin/clang++ accepts -g... no

checking dependency style of ~/clang-3.3/bin/clang++... none

checking whether ~/clang-3.3/bin/clang++ supports C++11 features by default... no

checking whether ~/clang-3.3/bin/clang++ supports C++11 features with -std=c++11... no

checking whether ~/clang-3.3/bin/clang++ supports C++11 features with -std=c++0x... no

configure: error: *** A compiler with support for C++11 language features is required.

I also tried installing Clang 3.3 via homebrew:

brew install llvm --with-clang --with-asan --all-targets --rtti --universal --with-python

.... 

./configure CXX=/usr/local/bin/clang++

checking for a BSD-compatible install... /usr/bin/install -c

checking whether build environment is sane... yes

checking for a thread-safe mkdir -p... build-aux/install-sh -c -d

checking for gawk... no

checking for mawk... no

checking for nawk... no

checking for awk... awk

checking whether make sets $(MAKE)... yes

checking whether make supports nested variables... yes

checking for gcc... gcc

checking whether the C compiler works... yes

checking for C compiler default output file name... a.out

checking for suffix of executables...

checking whether we are cross compiling... no

checking for suffix of object files... o

checking whether we are using the GNU C compiler... yes

checking whether gcc accepts -g... yes

checking for gcc option to accept ISO C89... none needed

checking for style of include used by make... GNU

checking dependency style of gcc... gcc3

checking whether we are using the GNU C++ compiler... yes

checking whether /usr/local/opt/llvm/bin/clang accepts -g... yes

checking dependency style of /usr/local/opt/llvm/bin/clang... gcc3

checking whether /usr/local/opt/llvm/bin/clang supports C++11 features by default... no

checking whether /usr/local/opt/llvm/bin/clang supports C++11 features with -std=c++11... yes

checking whether /usr/local/opt/llvm/bin/clang -std=c++11 supports C++11 library features by default... no

checking whether /usr/local/opt/llvm/bin/clang -std=c++11 supports C++11 library features with -stdlib=libc++... no

configure: error: *** A C++ library support for C++11 features is required.

That seems to get a little closer, given that configure at least detects C++11 features.

Any ideas?

-Charles

[Kenton Varda]

For the non-homebrew version, it looks to me like your shell failed to expand "~" to "$HOME".  Are you perhaps not using bash?  Maybe I should change the instructions to use $HOME instead.

For the homebrew version, my guess is that you either need to tell homebrew to install a recent version of gcc (to get a recent libstdc++) or to explicitly install libc++ (which is LLVM's alternative to libstdc++).  I haven't used homebrew so I don't know the exact steps here.

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
Visit this group at http://groups.google.com/group/capnproto.
 
 

[Charles Strahan]

Unfortunately, I get the same error if I use the full path to the copy in my home directory. Perhaps there's something different in our environment that has this working for you?

It appears the homebrew installed llvm does not (yet) pass the --enable-libcpp flag to configure:

https://github.com/mxcl/homebrew/pull/19914

https://github.com/mxcl/homebrew/pull/20251

Hopefully that'll be resolved soon, making everything a lot easier for OSX devs.

In the meantime, I'll just use a GNU/Linux vm, and continue writing my Ruby c extension there :).

(BTW, thanks for building such an awesome library!)

-Charles

[Kenton Varda]

On Thu, Aug 1, 2013 at 7:58 AM, Charles Strahan <charles....@gmail.com> wrote:

Unfortunately, I get the same error if I use the full path to the copy in my home directory. Perhaps there's something different in our environment that has this working for you?

I suspect you are now getting a slightly different error that happens to look similar.  :)  This line makes it pretty clear that your clang binary wasn't working at all:

    checking whether ~/clang-3.3/bin/clang++ accepts -g... no

(Of course, the line also makes it clear that the reason it wasn't working is because ~ wasn't properly expanded.)

Anyway, the configure script creates a file called config.log which contains more details on what went wrong.  Can you send it to me?

 

It appears the homebrew installed llvm does not (yet) pass the --enable-libcpp flag to configure:

https://github.com/mxcl/homebrew/pull/19914

https://github.com/mxcl/homebrew/pull/20251

Hopefully that'll be resolved soon, making everything a lot easier for OSX devs.

In the meantime, I'll just use a GNU/Linux vm,

Another option that has been reported to work is installing gcc 4.7+ via macports.

 

and continue writing my Ruby c extension there :).

Sweet!  :D

[Charles Strahan]

I suspect you are now getting a slightly different error that happens to look similar.  :)  This line makes it pretty clear that your clang binary wasn't working at all:

Good call!

checking for a BSD-compatible install... /usr/bin/install -c

checking whether build environment is sane... yes

checking for a thread-safe mkdir -p... build-aux/install-sh -c -d

checking for gawk... no

checking for mawk... no

checking for nawk... no

checking for awk... awk

checking whether make sets $(MAKE)... yes

checking whether make supports nested variables... yes

checking for gcc... gcc

checking whether the C compiler works... yes

checking for C compiler default output file name... a.out

checking for suffix of executables...

checking whether we are cross compiling... no

checking for suffix of object files... o

checking whether we are using the GNU C compiler... yes

checking whether gcc accepts -g... yes

checking for gcc option to accept ISO C89... none needed

checking for style of include used by make... GNU

checking dependency style of gcc... gcc3

checking whether we are using the GNU C++ compiler... yes

checking whether /Users/charlesstrahan/clang-3.3/bin/clang++ accepts -g... yes

checking dependency style of /Users/charlesstrahan/clang-3.3/bin/clang++... gcc3

checking whether /Users/charlesstrahan/clang-3.3/bin/clang++ supports C++11 features by default... no

checking whether /Users/charlesstrahan/clang-3.3/bin/clang++ supports C++11 features with -std=c++11... yes

checking whether /Users/charlesstrahan/clang-3.3/bin/clang++ -std=c++11 supports C++11 library features by default... no

checking whether /Users/charlesstrahan/clang-3.3/bin/clang++ -std=c++11 supports C++11 library features with -stdlib=libc++... no

configure: error: *** A C++ library support for C++11 features is required.

I've attached the config.log file. Thanks for the help!

[Kenton Varda]

OK, so the problem now is that it's not finding libc++.  Is it possible you missed the step of adding the symlink to your clang tree?

    ln -s /usr/lib/c++ $HOME/clang-3.3/lib/c++

[Charles Strahan]

Well, I did initially symlink, and then got desperate and just cp'd it over.

I symlinked to c++ this time around, and attached the new config.log:

λ ln -s /usr/lib/c++ $HOME/clang-3.3/lib/c++

λ ls -la $HOME/clang-3.3/lib/ | grep c++

lrwxr-xr-x    1 charlesstrahan  staff        12 Aug  1 11:36 c++ -> /usr/lib/c++

[Kenton Varda]

You know, looking at the error, it looks to me like this might actually be a bug in the version of libc++ that ships with XCode.  The error seems to happen in code that is using a new C++11 feature that was just added in Clang 3.3.  I assume the code gets macro'd out in Clang 3.2.  Maybe the libc++ that comes with XCode is simply too old to work with Clang 3.3?  I haven't actually tried compiling with 3.3 yet, only 3.2, but I'll try it before the next release.

[Charles Strahan]

Actually, I just tried the instructions you give for using Clang 3.2, and it appears to work if I use $HOME instead of ~. Dunno why I didn't consider trying that earlier - I guess I figured 3.3 should have worked just was well, but I guess not.

Just to be complete, I've attached the new log.

If you're curious, I use zshell, which apparently (either out of the box, or due to my configuration) doesn't expand ~.

Thanks for all the help!

[Kenton Varda]

Yep, that's what I figured.  It sounds like the latest XCode is too old for the latest Clang.  Grr.  I'll have to mention this in the docs.

[text...@gmail.com]

I would appreciate exact step-by-step as I can’t build this on OS X. The steps posted show:

1. ln -s /usr/lib/c++ ~/clang-3.2/lib/c++

uctions (also, /usr/lib/c++ already exist on my system).

I also tried with clang from MacPorts and gcc 4.8, but both of those failed as well.

Btw: trying to build without creating the above symlink gives me the following failure from ./configure:

checking dependency style of /Users/duff/Source/clang-3.2/bin/clang++... gcc3
checking whether /Users/duff/Source/clang-3.2/bin/clang++ supports C++11 features by default... no
checking whether /Users/duff/Source/clang-3.2/bin/clang++ supports C++11 features with -std=gnu++11... yes
checking whether /Users/duff/Source/clang-3.2/bin/clang++ -std=gnu++11 supports C++11 library features by default... no
checking whether /Users/duff/Source/clang-3.2/bin/clang++ -std=gnu++11 supports C++11 library features with -stdlib=libc++... no

[Kenton Varda]

Hi there,

On Fri, Aug 16, 2013 at 2:57 AM, <text...@gmail.com> wrote:

I would appreciate exact step-by-step as I can’t build this on OS X.

The web page has the exact step-by-step that works for me.  I'm not sure what else I can provide.

 

The steps posted show:

1. ln -s /usr/lib/c++ ~/clang-3.2/lib/c++

uctions (also, /usr/lib/c++ already exist on my system).

Yes, /usr/lib/c++ is supposed to already exist.  The command makes ~/clang-3.2/lib/c++ point to /usr/lib/c++.

It looks like part of your message got cut off.  What exactly went wrong with this command?

 

I also tried with clang from MacPorts and gcc 4.8, but both of those failed as well.

This is pretty vague.  Did you set CXX to point at the right compiler?  What was the error message?

 

Btw: trying to build without creating the above symlink gives me the following failure from ./configure:

checking dependency style of /Users/duff/Source/clang-3.2/bin/clang++... gcc3
checking whether /Users/duff/Source/clang-3.2/bin/clang++ supports C++11 features by default... no
checking whether /Users/duff/Source/clang-3.2/bin/clang++ supports C++11 features with -std=gnu++11... yes
checking whether /Users/duff/Source/clang-3.2/bin/clang++ -std=gnu++11 supports C++11 library features by default... no
checking whether /Users/duff/Source/clang-3.2/bin/clang++ -std=gnu++11 supports C++11 library features with -stdlib=libc++... no

Yes, this is exactly what would happen without the symlink.

-Kenton

[Allan Odgaard]

On 16 Aug 2013, at 12:50, Kenton Varda wrote:

> […] /usr/lib/c++ is supposed to already exist. The command makes
> ~/clang-3.2/lib/c++ point to /usr/lib/c++.

Ah, I had it the other way around. The text says “Xcode’s libstdc++
[…] is too old” so I was expecting to find a newer libc++ in
clang-3.2 that I had to make /usr/lib/c++ point to.

>> I also tried with clang from MacPorts and gcc 4.8, but both of those
>> failed as well.
> This is pretty vague. Did you set CXX to point at the right compiler?
> What was the error message?

For clang 3.2 and 3.4-svn I got a compiler stack dump indicating a bug
in clang (or a problem with my build of clang).

For gcc 4.8 I got the linker error quoted below.

Anyway, I got it working using the clang 3.2 binary distro, after
creating the proper symbolic link.

Thanks for your quick reply.

Undefined symbols for architecture x86_64:
"std::__1::__vector_base_common<true>::__throw_length_error()
const", referenced from:
std::__1::vector<kj::ArrayPtr<capnp::word const>,
std::__1::allocator<kj::ArrayPtr<capnp::word const> >
>::__append(unsigned long) in arena.o
void std::__1::vector<kj::Own<capnp::_::SegmentBuilder>,
std::__1::allocator<kj::Own<capnp::_::SegmentBuilder> >
>::__push_back_slow_path<kj::Own<capnp::_::SegmentBuilder>
>(kj::Own<capnp::_::SegmentBuilder>&&) in arena.o
"std::__1::__next_prime(unsigned long)", referenced from:
std::__1::__hash_table<std::__1::pair<unsigned int,
kj::Own<capnp::_::SegmentReader> >,
std::__1::__unordered_map_hasher<unsigned int,
kj::Own<capnp::_::SegmentReader>, std::__1::hash<unsigned int>, true>,
std::__1::__unordered_map_equal<unsigned int,
kj::Own<capnp::_::SegmentReader>, std::__1::equal_to<unsigned int>,
true>, std::__1::allocator<std::__1::pair<unsigned int,
kj::Own<capnp::_::SegmentReader> > > >::rehash(unsigned long) in arena.o
ld: symbol(s) not found for architecture x86_64

[Kenton Varda]

On Fri, Aug 16, 2013 at 4:39 AM, Allan Odgaard <text...@gmail.com> wrote:

On 16 Aug 2013, at 12:50, Kenton Varda wrote:

[…] /usr/lib/c++ is supposed to already exist.  The command makes
~/clang-3.2/lib/c++ point to /usr/lib/c++.

Ah, I had it the other way around. The text says “Xcode’s libstdc++ […] is too old” so I was expecting to find a newer libc++ in clang-3.2 that I had to make /usr/lib/c++ point to.

Yeah, the problem is that C++ standard library authors have horrible taste in names.  So, there are two competing C++ standard libraries:  libstdc++ and libc++.  Xcode's libstdc++ is too old, but its libc++ is new enough.  The symlink is to make Clang see libc++.

For clang 3.2 and 3.4-svn I got a compiler stack dump indicating a bug in clang (or a problem with my build of clang).

Can you send me the error, out of curiosity?

Anyway, I got it working using the clang 3.2 binary distro, after creating the proper symbolic link.

Great!

 

    Undefined symbols for architecture x86_64:
      "std::__1::__vector_base_common<true>::__throw_length_error() const", referenced from:
          std::__1::vector<kj::ArrayPtr<capnp::word const>, std::__1::allocator<kj::ArrayPtr<capnp::word const> > >::__append(unsigned long) in arena.o
          void std::__1::vector<kj::Own<capnp::_::SegmentBuilder>, std::__1::allocator<kj::Own<capnp::_::SegmentBuilder> > >::__push_back_slow_path<kj::Own<capnp::_::SegmentBuilder> >(kj::Own<capnp::_::SegmentBuilder>&&) in arena.o
      "std::__1::__next_prime(unsigned long)", referenced from:
          std::__1::__hash_table<std::__1::pair<unsigned int, kj::Own<capnp::_::SegmentReader> >, std::__1::__unordered_map_hasher<unsigned int, kj::Own<capnp::_::SegmentReader>, std::__1::hash<unsigned int>, true>, std::__1::__unordered_map_equal<unsigned int, kj::Own<capnp::_::SegmentReader>, std::__1::equal_to<unsigned int>, true>, std::__1::allocator<std::__1::pair<unsigned int, kj::Own<capnp::_::SegmentReader> > > >::rehash(unsigned long) in arena.o
    ld: symbol(s) not found for architecture x86_64

Eek, this seems like a bug in the standard libraries.

-Kenton

[Derrick Johnson]

Hi Kenton

Your instructions for OS X worked great.  I am, however, having troubles linking on Linux!

I have one schema file (test.capnp) and one C++ file (main.cpp).  I'm using CapNPC 0.2.1.

I'm using g++ 4.7.2 on Ubuntu 12.10.

Here is test.capnp:

@0xaacca5cd67e6d5a7;

using Cxx = import "/capnp/c++.capnp";

$Cxx.namespace("data");

struct Thing

{

value @0 :Int64;

}

Here is main.cpp:

#include "test.capnp.h"   

#include "capnp/message.h"

int main()

{

::capnp::MallocMessageBuilder msg;

data::Thing::Builder thing = msg.initRoot<data::Thing>();

thing.setValue(1234);

return 0;

}

Here is how I compile test.capnp (I installed CapNProto in ~/usr instead of /usr using --prefix with configure script):

capnpc -o c++ -I ~/usr/include test.capnp   (I get no errors here... all's well)

Here is how I build main.cpp and the linker errors I get (I'm using g++ 4.7.2):

g++ -std=c++11 -I ~/usr/include ~/usr/lib/libcapnp.a main.cpp test.capnp.c++

/tmp/cccHZAZb.o: In function `main':

main.cpp:(.text+0x21): undefined reference to `capnp::MallocMessageBuilder::MallocMessageBuilder(unsigned int, capnp::AllocationStrategy)'

main.cpp:(.text+0x62): undefined reference to `capnp::MallocMessageBuilder::~MallocMessageBuilder()'

main.cpp:(.text+0x78): undefined reference to `capnp::MallocMessageBuilder::~MallocMessageBuilder()'

/tmp/cccHZAZb.o: In function `data::Thing::Builder capnp::MessageBuilder::initRoot<data::Thing>()':

main.cpp:(.text._ZN5capnp14MessageBuilder8initRootIN4data5ThingEEENT_7BuilderEv[_ZN5capnp14MessageBuilder8initRootIN4data5ThingEEENT_7BuilderEv]+0x53): undefined reference to `capnp::MessageBuilder::initRoot(capnp::_::StructSize)'

collect2: error: ld returned 1 exit status

Again, I got everything to work fine on OS X so I think my .capnp file and main.cpp are fine...

Thanks!

On Friday, June 14, 2013 6:35:51 PM UTC-4, Kenton Varda wrote:

[Derrick Johnson]

BTW You might have more luck using /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/c++ as the libc++ stuff for Clang.

Just symlink that directory instead of /usr/lib/c++ for <your clang dir>/lib/c++

There's a bug in /usr/lib/c++ that affects std::atomic on OS X under Clang 3.2, for example, so this fix might also fix your issue.

I found this to support it: http://stackoverflow.com/questions/16215421/clang-3-2-fails-on-stdatomic-a-libc-issue

[Kenton Varda]

Hi Derrick,

On Thu, Aug 22, 2013 at 6:46 PM, Derrick Johnson <badma...@gmail.com> wrote:

g++ -std=c++11 -I ~/usr/include ~/usr/lib/libcapnp.a main.cpp test.capnp.c++

The problem is the order of arguments.  The linker only looks for symbols in the libraries listed after the thing using those symbol.  If you move libcapnp.a to the end of the command, it should work.

I'm not really sure why the linker works this way, but there you go...  :/

-Kenton

[Derrick Johnson]

Thanks. Also can you provide a quick example of how to create an object using an existing buffer that I supply? For example, one that is in the stack rather than in the heap? FlatMessageBuilder I cannot get to work... Thx!

[Kenton Varda]

On Fri, Aug 23, 2013 at 6:49 AM, Derrick Johnson <badma...@gmail.com> wrote:

Thanks.  Also can you provide a quick example of how to create an object using an existing buffer that I supply?  For example, one that is in the stack rather than in the heap?  FlatMessageBuilder I cannot get to work... Thx!

MallocMessageBuilder allows you to pass a scratch array as its first parameter, and it will then use that space first before falling back to malloc.  It should be straightforward to use:

  capnp::word scratch[128];

  memset(scratch, 0, sizeof(scratch));

  capnp::MallocMessageBuilder message(kj::arrayPtr(scratch, 128););

  auto root = message.initRoot<MyStruct>();

  root.setFoo(123);

  root.setBar(456);

  writeMessageToFd(fd, message);

FlatMessageBuilder should work as well, but is only safe if you're absolutely sure that your message will not end up taking more space than you provide it.