Building skydb unstable

[Edward Middleton]

I noticed there is a lot of cool stuff going into sky unstable branch .
I noticed the Makefile has been remove. How do you currently build
unstable?

Edward

[Ben Johnson]

hey Edward-

Shopify sponsored development of the multinode clustering so I'm out in Ottawa working with their team right now. They're switching away from Mondrian to use Sky as their reporting backend so they've been putting it through its paces.

Sky was quiet for a little while but that was mostly because I was working on a Go implementation of the Raft distributed consensus protocol:

https://github.com/benbjohnson/go-raft

I also switched out the LevelDB backend for LMDB. There are concurrency issues with LevelDB when using the C API (unless you memcpy every key/value when you read) so I had to find a different backing store. LMDB has been working great and it's fast as hell. :)

As for building Sky, it's easier now:

1. Follow the instructions on the sky-deps repo: https://github.com/skydb/sky-deps/tree/unstable

2. Run: go get github.com/skydb/sky

3. Run: ./skyd

Go 1.1 let me embed csky into Go so there's not a separate build step for it.

Let me know if you have any questions. There's a lot going into Sky lately. There's definitely some cool stuff coming up.

Ben

[Daniel Fagnan]

Has these instructions changed?

> can't load package: /Users/daniel/sky-deps/src/github.com/skydb/sky/skyd.go:4:2: local import "./skyd" in non-local package

I'm getting this error during those instructions.

[Ben Johnson]

Daniel-

Actually, yes, it looks like those instructions have changed. I recently did some clean up on the Sky packages. Here's the new instructions:

1. Follow the instructions on the sky-deps repo: https://github.com/skydb/sky-deps/tree/unstable

2. Make sure you have $GOPATH set up correctly and then run the following:

$ go get github.com/skydb/sky

$ cd $GOPATH/src/github.com/skydb/sky

$ git checkout unstable

$ cd skyd

$ go install

$ skyd

Once the unstable branch is released the instructions will be much simpler. Let me know if you have any problems with that.

Ben Johnson

b...@skylandlabs.com

[Daniel Fagnan]

Awesome, thanks. I had to manually install 3rd-party packages, as well as building LMDB for the wrapper package to work, so just a heads up for anyone else trying to build it.

[Joshua Bowles]

I ran into a problem compiling gomdb, which is a blocker for me getting up and running with skydb. Once I'm sure it is not some issue with me or my env will open an issue with gomdb:

Here is my error (on Mac osX 10.9)

# github.com/szferi/gomdb

clang: error: argument unused during compilation: '-fno-eliminate-unused-debug-types'

AND GCC INFO:

gcc --version                                                                               

Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/c++/4.2.1

Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)

Target: x86_64-apple-darwin13.0.0

gcc-42 --version

i686-apple-darwin11-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5666) (dot 3)

[Ben Johnson]

Hey Joshua-

What version of Go are you using?

Ben

[Joshua Bowles]

go version go1.1.1 darwin/amd64

[Joshua Bowles]

an update for anyone interested:

Looks like

   # github.com/szferi/gomdb

   clang: error: argument unused during compilation: '-fno-eliminate-unused-debug-types'

is an issue between Go and Clang: https://code.google.com/p/go/issues/detail?id=4829. It is supposed to be fixed in Go 1.2 (which still only a release candidate go1.2-rc3), but it has not been backported to the stable release Go 1.1.2. I've closed my issue on gomdb https://github.com/szferi/gomdb/issues/10

Last night I was able to install/compile gomdb, sky-deps, and skydb (unstable) on a CentOS6 machine. :)

[Joshua Bowles]

Lastly, I can also confirm that go 1.1.2 won't work either, BUT after upgrading to go1.2-rc3 on Mac OSX 10.9 (Mavericks)

   go version go1.2rc3 darwin/amd64

and following instructions above for sky-deps and skydb (unstable), everything is good.

There is a permissions warning about /var/run/skyd.pid; I can either change permissions for it or just run skyd as sudo.

$sudo skyd           

2013/11/07 05:30:04 Sky v0.4.0 (unstable) is now listening on http://localhost:8585

2013/11/07 05:30:36 [::1]:51483 "GET / HTTP/1.1" 200 0.000

[Joshua Bowles]

Aagh! Sorry for spam, but as I read more, it looks like you may be able to build with Go 1.1.2. If this effects anyone read through this bug report for Go https://code.google.com/p/go/issues/detail?id=6544 and this issue thread for Homebrew build recipe of go https://github.com/mxcl/homebrew/issues/23119.

I'm not looking into this anymore, I don't mind running go1.2-rc3

[Ben Johnson]

Josh-

Good find. I was stumped as to why that was happening. I added a note to the README so other people can find it.

Thanks!

Ben

[sant...@revenuemantra.com]

Hi Ben,

      

Thanks for the info. I had some issues during the last step go install. And the issues are like:

config/config.go:4:2: cannot find package "github.com/BurntSushi/toml" in any of:
    /usr/lib/go/src/pkg/github.com/BurntSushi/toml (from $GOROOT)
    /home/rmtwo/gocode/src/github.com/BurntSushi/toml (from $GOPATH)
../db/cursor_slice.go:3:8: cannot find package "github.com/szferi/gomdb" in any of:
    /usr/lib/go/src/pkg/github.com/szferi/gomdb (from $GOROOT)
    /home/rmtwo/gocode/src/github.com/szferi/gomdb (from $GOPATH)
../core/event.go:6:2: cannot find package "github.com/ugorji/go/codec" in any of:
    /usr/lib/go/src/pkg/github.com/ugorji/go/codec (from $GOROOT)
    /home/rmtwo/gocode/src/github.com/ugorji/go/codec (from $GOPATH)

Could you please tell me where I go wrong. Thank you
 

[Ben Johnson]

It looks like you need to run "go get github.com/skydb/sky/skyd" to get the dependencies.

Also, there's a Vagrant setup for Sky here that might be helpful:

https://github.com/skydb/vagrant/tree/unstable

It does the whole installation of LMDB and LuaJIT 2.0.1 to get you up and running on an Ubuntu instance in Vagrant and has worked well for me so far.

The install process should be much easier once the next version of Sky (v0.4.0) is released. Building non-master branch Go programs is a pain.

Ben