Error during bootstrap.sh

[Anantha]

Hi,

When I try to install Vitess, bootstrap.sh gives the following error. I followed the instructions on https://github.com/youtube/vitess/blob/master/doc/GettingStarted.md and got all the dependencies working. 

src/base/linuxthreads.cc: In function ‘void ListerThread(ListerParams*)’:
src/base/linuxthreads.cc:312:22: error: invalid conversion from ‘void (*)(int, siginfo_t*, void*)’ to ‘void (*)(int, siginfo*, void*)’ [-fpermissive]
 sa.sa_sigaction_ = SignalHandler;
 ^
make[1]: *** [libtcmalloc_la-linuxthreads.lo] Error 1
make[1]: *** Waiting for unfinished jobs....
In file included from src/heap-checker.cc:78:0:
src/heap-checker.cc: In static member function ‘static int HeapLeakChecker::IgnoreLiveThreadsLocked(void*, int, pid_t*, __va_list_tag*)’:
src/heap-checker.cc:1017:22: warning: typedef ‘SP_register_does_not_look_like_a_pointer’ locally defined but not used [-Wunused-local-typedefs]
 SP_register_does_not_look_like_a_pointer);
 ^
src/base/basictypes.h:188:39: note: in definition of macro ‘COMPILE_ASSERT’
 typedef CompileAssert<(bool(expr))> msg[bool(expr) ? 1 : -1]
 ^
libtool: compile: g++ -DHAVE_CONFIG_H -I. -I. -I./src -I./src -pthread -DNDEBUG -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -fno-builtin-malloc -fno-builtin-free -fno-builtin-realloc -fno-builtin-calloc -fno-builtin-cfree -fno-builtin-memalign -fno-builtin-posix_memalign -fno-builtin-valloc -fno-builtin-pvalloc -Wno-unused-result -g2 -O2 -fno-strict-aliasing -fno-omit-frame-pointer -I/home/ubuntu/workspace/src/github.com/youtube/vitess/third_party/mysql/ncurses/include -I/home/ubuntu/workspace/src/github.com/youtube/vitess/third_party/mysql/ncurses/include -DHAVE_TERM_H -fno-rtti -MT libtcmalloc_la-tcmalloc.lo -MD -MP -MF .deps/libtcmalloc_la-tcmalloc.Tpo -c src/tcmalloc.cc -fPIC -DPIC -o libtcmalloc_la-tcmalloc.o >/dev/null 2>&1
libtool: compile: g++ -DHAVE_CONFIG_H -I. -I. -I./src -I./src -pthread -DNDEBUG -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -fno-builtin-malloc -fno-builtin-free -fno-builtin-realloc -fno-builtin-calloc -fno-builtin-cfree -fno-builtin-memalign -fno-builtin-posix_memalign -fno-builtin-valloc -fno-builtin-pvalloc -Wno-unused-result -g2 -O2 -fno-strict-aliasing -fno-omit-frame-pointer -I/home/ubuntu/workspace/src/github.com/youtube/vitess/third_party/mysql/ncurses/include -I/home/ubuntu/workspace/src/github.com/youtube/vitess/third_party/mysql/ncurses/include -DHAVE_TERM_H -fno-rtti -MT libtcmalloc_la-heap-checker.lo -MD -MP -MF .deps/libtcmalloc_la-heap-checker.Tpo -c src/heap-checker.cc -fPIC -DPIC -o libtcmalloc_la-heap-checker.o >/dev/null 2>&1
libtool: compile: g++ -DHAVE_CONFIG_H -I. -I. -I./src -I./src -DNO_TCMALLOC_SAMPLES -pthread -DNDEBUG -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -fno-builtin-malloc -fno-builtin-free -fno-builtin-realloc -fno-builtin-calloc -fno-builtin-cfree -fno-builtin-memalign -fno-builtin-posix_memalign -fno-builtin-valloc -fno-builtin-pvalloc -Wno-unused-result -DTCMALLOC_FOR_DEBUGALLOCATION -g2 -O2 -fno-strict-aliasing -fno-omit-frame-pointer -I/home/ubuntu/workspace/src/github.com/youtube/vitess/third_party/mysql/ncurses/include -I/home/ubuntu/workspace/src/github.com/youtube/vitess/third_party/mysql/ncurses/include -DHAVE_TERM_H -fno-rtti -MT libtcmalloc_minimal_debug_la-debugallocation.lo -MD -MP -MF .deps/libtcmalloc_minimal_debug_la-debugallocation.Tpo -c src/debugallocation.cc -fPIC -DPIC -o libtcmalloc_minimal_debug_la-debugallocation.o >/dev/null 2>&1
make: *** [all-recursive] Error 1

Is this an error with the Vitess library or am I doing something wrong while installing it?

Thanks.

[Anthony Yeh]

These look like the errors that happen when trying to build Google MySQL 5.1 on Ubuntu Trusty. Did you intend to use MariaDB 10 instead? If so, make sure to "export MYSQL_FLAVOR=MariaDB" before running bootstrap.sh. Keep in mind that MYSQL_FLAVOR is case-sensitive. You may need to start in a fresh shell and working directory to get rid of env vars and files created during bootstrap in Google MySQL 5.1 mode.

Hope that helps.

--
You received this message because you are subscribed to the Google Groups "vitess" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vitess+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Anantha]

Thanks for your reply.

I completely uninstalled MySQL, and then installed MariaDB 10.0.14. I also exported the MYSQL_FLAVOR. But the error still persists.

[Anthony Yeh]

Could you upload a log of all the commands you run and their outputs on something like https://gist.github.com/ ?

[Anantha]

I started over and tried the installation again. This time, the ./bootstrap worked, but make failed with the following error: 

go/rpcplus/pbrpc/client.go:10:2: cannot find package "code.google.com/p/goprotobuf/proto" in any of:

        /usr/src/pkg/code.google.com/p/goprotobuf/proto (from $GOROOT)

        /home/ubuntu/workspace/src/code.google.com/p/goprotobuf/proto (from $GOPATH)

        /home/ubuntu/go/src/code.google.com/p/goprotobuf/proto

make: *** [build] Error 1

Here is the gist file of my installation process, if you need it: https://gist.github.com/anonymous/3bd082c00ea8bdcd4048/e5beea928a083f75781bf0acf542488ab1137faf

[Anthony Yeh]

There may have been a step that failed in bootstrap. Try this from the vitess git root and see if it produces any errors:

$ source dev.env

$ go get code.google.com/p/goprotobuf/proto

[Anantha]

I got this error:

load cmd/cgo: package cmd/cgo: open /usr/src/cmd/cgo: no such file or directory

make: *** [build] Error 1

My $GOPATH is set to $HOME/go.

...

[Anthony Yeh]

Did you install Go into somewhere other than /usr/local/go? For instance if you installed the Go distribution into $HOME/go, you should do:

$ export GOROOT=$HOME/go

[Anantha]

No. Go is installed in /usr/local/go.

</bl

...

[Sugu Sougoumarane]

It looks like a go install problem. I'm wondering if you can get a quick answer if you posted the issue to https://groups.google.com/forum/#!forum/golang-nuts.

[Anantha Nithya]

Ok. Thanks a lot. Much appreciated.

[Sugu Sougoumarane]

Were you able to make progress after their suggestions?

[Anantha]

I started over a couple of times, and still was not able to solve the error during bootstrap.sh (the first error with which I started this topic). But when I try ./bootstrap.sh again, it completes without errors. Then I do a 'source dev.env', and do a 'make build'. 

During make build, I get many errors like: 

go/acl/acl.go:17:2: cannot find package "github.com/golang/glog" in any of:

        /usr/src/pkg/github.com/golang/glog (from $GOROOT)

        ($GOPATH not set)

go/bson/marshal.go:14:2: cannot find package "github.com/youtube/vitess/go/bytes2" in any of:

        /usr/src/pkg/github.com/youtube/vitess/go/bytes2 (from $GOROOT)

        ($GOPATH not set)

go/bson/unmarshal_util.go:14:2: cannot find package "github.com/youtube/vitess/go/hack" in any of:

        /usr/src/pkg/github.com/youtube/vitess/go/hack (from $GOROOT)

        ($GOPATH not set)

go/cmd/mysqlctl/mysqlctl.go:16:2: cannot find package "github.com/youtube/vitess/go/vt/dbconfigs" in any of:

        /usr/src/pkg/github.com/youtube/vitess/go/vt/dbconfigs (from $GOROOT)

        ($GOPATH not set)

go/cmd/mysqlctl/mysqlctl.go:17:2: cannot find package "github.com/youtube/vitess/go/vt/key" in any of:

        /usr/src/pkg/github.com/youtube/vitess/go/vt/key (from $GOROOT)

        ($GOPATH not set)

go/cmd/mysqlctl/mysqlctl.go:18:2: cannot find package "github.com/youtube/vitess/go/vt/logutil" in any of:

        /usr/src/pkg/github.com/youtube/vitess/go/vt/logutil (from $GOROOT)

        ($GOPATH not set)

Here are some of my relevant my environment variables:

VTDATAROOT=/home/ubuntu/gocode/vtdataroot

MYSQL_FLAVOR=MariaDB

VTTOP=/home/ubuntu/gocode/src/github.com/youtube/vitess

GOBIN=/home/ubuntu/gocode/bin

WORKSPACE=/home/ubuntu/gocode/src/github.com/youtube/vitess

CGO_LDFLAGS= -L/home/ubuntu/gocode/dist/vt-zookeeper-3.3.5/lib

VTROOT=/home/ubuntu/gocode

LD_LIBRARY_PATH=/home/ubuntu/gocode/dist/vt-zookeeper-3.3.5/lib:/usr/lib:/home/ubuntu/gocode/dist/vt-zookeeper-3.3.5/lib/

GOTOP=/home/ubuntu/gocode/src/github.com/youtube/vitess/go

PATH=/home/ubuntu/gocode/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/ubuntu/go/bin

CGO_CFLAGS= -I/home/ubuntu/gocode/dist/vt-zookeeper-3.3.5/include/c-client-src

PWD=/home/ubuntu/gocode/src/github.com/youtube/vitess

GOARCH=amd64

VT_MYSQL_ROOT=/usr

VTPORTSTART=15000

HOME=/home/ubuntu

GOROOT=/home/ubuntu/go

GOOS=linux

PYTHONPATH=/home/ubuntu/gocode/py-vtdb:/home/ubuntu/gocode/dist/py-cbson/lib/python2.7/site-packages:/home/ubuntu/gocode/dist/py-vt-bson-0.3.2/lib/python2.7/site-packages

LOGNAME=ubuntu

PYTOP=/home/ubuntu/gocode/src/github.com/youtube/vitess/py

PKG_CONFIG_PATH=/home/ubuntu/gocode/lib

GOPATH=/home/ubuntu/gocode

Thanks in advance.

...

[Anthony Yeh]

It's strange that it seems to be assuming GOROOT=/usr and it says GOPATH is not set, despite them being set to something else in the env dump you pasted.

Can you provide some information from the output of these commands?

which go

alias go

go version

go env

echo $0

--

You received this message because you are subscribed to the Google Groups "vitess" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vitess+unsubscribe@googlegroups.com.

[Anantha]

Here is the output:

ubuntu@ubuntu4:~$ which go

/usr/bin/go

ubuntu@ubuntu4:~$ alias go

-bash: alias: go: not found

ubuntu@ubuntu4:~$ go version

go version xgcc (Ubuntu 4.9.1-0ubuntu1) 4.9.1 linux/amd64

ubuntu@ubuntu4:~$ go env

GOARCH="amd64"

GOBIN="/home/ubuntu/go/bin"

GOCHAR="6"

GOEXE=""

GOHOSTARCH="amd64"

GOHOSTOS="linux"

GOOS="linux"

GOPATH="/home/ubuntu/gocode"

GORACE=""

GOROOT="/home/ubuntu/go"

GOTOOLDIR="/home/ubuntu/go/pkg/tool/linux_amd64"

TERM="dumb"

CC="gcc"

GOGCCFLAGS="-g -O2 -fPIC -m64 -pthread"

CXX="g++"

CGO_ENABLED="1"

ubuntu@ubuntu4:~$ echo $0

-bash

echo $0

libtool: compile: g++ -DHAVE_CONFIG_H -I. -I. -I./src -I./src -pthread -DNDEBUG -Wall -Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -fno-builtin-malloc -fno-builtin-free -fno-builtin-realloc -fno-builtin-calloc -fno-builtin-cfree -fno-builtin-memalign -fno-builtin-posix_memalign -fno-builtin-valloc -fno-builtin-pvalloc -Wno-unused-result -g2 -O2 -fno-strict-aliasing -fno-omit-frame-pointer -I/home/ubuntu/workspace/src/github.com/youtube/vitess/third_party/mysql/ncurses/include -I/home/ubuntu/workspace/src/github.com/youtube/vitess/third_party/mysql/ncurses/include -DHAVE_TERM_H -fno-rtti -MT libtcmalloc_la-heap-checker.lo -MD <span style="color:#6

...

[Anthony Yeh]

Ah it looks like you have gccgo installed and it is taking precedence in your $PATH. If you can, I recommend removing gccgo and only installing the gc toolchain (the one from golang.org). You could also keep both if you change your $PATH so that the golang.org version of the "go" command gets found before the one in /usr/bin.

--

You received this message because you are subscribed to the Google Groups "vitess" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vitess+un...@googlegroups.com.

[Anantha]

I removed gccgo, and set my path to look like:

/home/ubuntu/gocode/bin:/home/ubuntu/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

The Hello World program seems to be working correctly. But I get this error during bootstrap.sh:

ubuntu@ubuntu4:~/gocode/src/github.com/youtube/vitess$ sudo ./bootstrap.sh 

./dev.env: line 55: go: command not found

skipping zookeeper build

./bootstrap.sh: line 38: go: command not found

./bootstrap.sh: line 40: go: command not found

./bootstrap.sh: line 41: go: command not found

./bootstrap.sh: line 42: go: command not found

Found MariaDB installation in /usr.

skipping bson python build

skipping cbson python build

creating git pre-commit hooks

source dev.env in your shell to complete the setup.

Is there anything else that I should change? 

Thanks for your time.

 typedef CompileAssert<(bool(expr))> msg[bool(expr<sp

...

[Anthony Yeh]

When you run a command in sudo, it doesn't pass on environment variables you set in your non-sudo shell, or in previous sudo commands. You shouldn't need to use sudo for building Vitess. It only installs files in the tree rooted at $VTROOT, which is where you were when you did the initial git clone.

--

[Anantha]

But I am working on a virtual machine, and without sudo, I get these "Permission Denied" messages. 

ubuntu@ubuntu4:~/gocode/src/github.com/youtube/vitess$ ./bootstrap.sh 

skipping zookeeper build

# cd .; hg clone -U https://code.google.com/p/go.tools /home/ubuntu/gocode/src/code.google.com/p/go.tools

abort: Permission denied: '/home/ubuntu/gocode/src/code.google.com/p/go.tools'

package code.google.com/p/go.tools/cmd/goimports: exit status 255

ln: cannot remove ‘/home/ubuntu/gocode/vthook/copy_snapshot_from_storage’: Permission denied

ln: cannot remove ‘/home/ubuntu/gocode/vthook/copy_snapshot_to_storage’: Permission denied

ln: cannot remove ‘/home/ubuntu/gocode/vthook/test.sh’: Permission denied

Can these messages be ignored?

src/heap-checker.cc: In static member function ‘static<span style="

...

[Anthony Yeh]

My guess is that you initially unpacked everything with sudo? Then all the files are owned by root. Could you try starting from scratch and not using sudo?

--

[Anantha]

Thanks a lot for all your suggestions. It works fine now.

...

[Anthony Yeh]

Great! Let us know if you have any more questions about running Vitess.

--

[Shrenik Parekh]

Hi Anthony et al,

I have installed Vitess on RHEL version 6. Go environment variables follows. I am getting following error while executing bootstrap.sh, help will be highly appreciate.

+ grpcio_ver=1.10.0

+ /vitess/dist/grpc/usr/local/bin/pip install --upgrade grpcio==1.10.0

DEPRECATION: Python 2.6 is no longer supported by the Python core team, please upgrade your Python. A future version of pip will drop support for Python 2.6

Collecting grpcio==1.10.0

/vitess/dist/grpc/usr/local/lib/python2.6/site-packages/pip/_vendor/urllib3/util/ssl_.py:339: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings

  SNIMissingWarning

/vitess/dist/grpc/usr/local/lib/python2.6/site-packages/pip/_vendor/urllib3/util/ssl_.py:137: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings

  InsecurePlatformWarning

  Downloading grpcio-1.10.0.tar.gz (14.0MB)

    100% |████████████████████████████████| 14.0MB 57kB/s

    Complete output from command python setup.py egg_info:

    Traceback (most recent call last):

      File "<string>", line 1, in <module>

      File "/tmp/pip-build-omWCLq/grpcio/setup.py", line 27, in <module>

        import sysconfig

    ImportError: No module named sysconfig

    ----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-omWCLq/grpcio/

You are using pip version 9.0.3, however version 10.0.0b2 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

ERROR: gRPC build failed

[root@sjlnxignfqc02 ~]# go version

go version go1.10.1 linux/amd64

[root@sjlnxignfqc02 ~]# which go

/usr/local/go/bin/go

[root@sjlnxignfqc02 ~]# go env

GOARCH="amd64"

GOBIN=""

GOCACHE="/root/.cache/go-build"

GOEXE=""

GOHOSTARCH="amd64"

GOHOSTOS="linux"

GOOS="linux"

GOPATH="/root/go"

GORACE=""

GOROOT="/usr/local/go"

GOTMPDIR=""

GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"

GCCGO="gccgo"

CC="gcc"

CXX="g++"

CGO_ENABLED="1"

CGO_CFLAGS="-g -O2"

CGO_CPPFLAGS=""

CGO_CXXFLAGS="-g -O2"

CGO_FFLAGS="-g -O2"

CGO_LDFLAGS="-g -O2"

PKG_CONFIG="pkg-config"

GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build924248703=/tmp/go-build"

[root@sjlnxignfqc02 ~]# which go

/usr/local/go/bin/go

[root@sjlnxignfqc02 ~]# alias go

-bash: alias: go: not found

[root@sjlnxignfqc02 ~]# go version

go version go1.10.1 linux/amd64

[root@sjlnxignfqc02 ~]# echo $0

-bash

[Shrenik Parekh]

Hi Anthony et el,

When I tried to execute, I am getting following errors.

# make build

go/acl/acl.go:29:2: cannot find package "github.com/golang/glog" in any of:

        /usr/local/go/src/github.com/golang/glog (from $GOROOT)

        /root/go/src/github.com/golang/glog (from $GOPATH)

go/cmd/automation_client/automation_client.go:26:2: cannot find package "github.com/golang/protobuf/proto" in any of:

        /usr/local/go/src/github.com/golang/protobuf/proto (from $GOROOT)

<blockquote style="margin:

[Sugu Sougoumarane]

Can you check if your GOPATH is set correctly, also your GOROOT?

I've seen these errors when I misconfigured those. I actually thought GOPATH was set by dev.env, but I could be wrong.

--

You received this message because you are subscribed to the Google Groups "vitess" group.

To unsubscribe from this group and stop receiving emails from it, send an email to vitess+unsubscribe@googlegroups.com.

[Shrenik Parekh]

Hi Sugu,

Thanks for your time, I tried several attempts by setting GOPATH and GOROOT, but still there is no success. I am wondering if you are aware of any document which depict, configure GO for Vitess which includes all pre-requisite for Vitess. My current environment variable setting follows, I have installed GO at /usr/local/go/bin. Let me know what should be the value for GOPATH and GOROOT. I will set and try it.

# echo $PATH

/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/root/bin:/usr/src/Python-2.7.14

[bin]# ./go version

go version go1.10.1 linux/amd64

Thanks 

Shrenik Parekh

To unsubscribe from this group and stop receiving emails from it, send an email to vitess+un...@googlegroups.com.

[Sugu Sougoumarane]

My guess is that the bootstrap didn't succeed successfully.

In my tree I can do this:

~/...vitess> ls $VTROOT/src/github.com/golang/protobuf

jsonpb  LICENSE  proto  protoc-gen-go  ptypes

~/...vitess>

Can you try this on yours? Also the '#' prompt makes me think you're running as root. People have had trouble installing vitess as root. If so, can you try as a regular user?

To unsubscribe from this group and stop receiving emails from it, send an email to vitess+unsubscribe@googlegroups.com.

[Shrenik Parekh]

Hi Sugu,

I executed bootstarp.sh using non root user mysql (Vitess is owned by user mysql), and getting following error for gRPC build failed.

 File "/vitess/dist/grpc/usr/local/lib/python2.6/site-packages/pip/_internal/cmdoptions.py", line 16, in <module>
    from pip._internal.index import (
  File "/vitess/dist/grpc/usr/local/lib/python2.6/site-packages/pip/_internal/index.py", line 526
    {str(c.version) for c in all_candidates},
                      ^
SyntaxError: invalid syntax
ERROR: gRPC build failed

Thanks

Shrenik Parekh

--

Thanks and regards.

Shrenik Parekh

[Michael Berlin]

Is this from the current master branch?

I recently made some changes to bootstrap.sh. Therefore, please check out the latest master branch first to rule out any old issues.

If the problem still persists, can you please provide some more details about your environment e.g. which OS?

[Shrenik Parekh]

Hi Michael,

I am using Red Hat Enterprise Linux Server release 6.6 (Santiago). I got Vitess using following command.

#git clone https://github.com/vitessio/vitess.git src/vitess.io/vitess

Thanks

Shrenik Parekh

[Michael Berlin]

Have you tried googling the error message?

It will be probably related to a super old Python version on your system.

We can also try to diagnose it further here:

Can you please add " -x" to the first line of bootstrap.sh i.e. change it to:

#!/bin/bash -x

and then post the output as attached .txt file here or send it to me directly?

This will make it easier to see where exactly the script is failing.

Thanks!