Deploying to Heroku Problems - go get (main redeclared in this block)

[Ollie Castle]

I am currently following the guides to deploy my app to Heroku (http://mmcgrana.github.com/2012/09/getting-started-with-go-on-heroku.html) but I cant find any examples using external libraries. When my app complies on Heroku I get errors from the external examples saying the examples directoires "main redeclared in this block". I have never seen this on my own machine and I cant find anything about it:

-----> Fetching custom git buildpack... done
-----> Go app detected
-----> Installing Go 1.0.3... done
       Installing Virtualenv... done
       Installing Mercurial... done
       Installing Bazaar... done
-----> Running: go get -tags heroku ./...
# code.google.com/p/log4go/examples
code.google.com/p/log4go/examples/FileLogWriter_Manual.go:17: main redeclared in this block
    previous declaration at code.google.com/p/log4go/examples/ConsoleLogWriter_Manual.go:9
code.google.com/p/log4go/examples/SimpleNetLogServer.go:21: main redeclared in this block
    previous declaration at code.google.com/p/log4go/examples/FileLogWriter_Manual.go:17
code.google.com/p/log4go/examples/SimpleNetLogServer.go:25: not enough arguments in call to net.ResolveUDPAddr
code.google.com/p/log4go/examples/SocketLogWriter_Manual.go:9: main redeclared in this block
    previous declaration at code.google.com/p/log4go/examples/SimpleNetLogServer.go:21
code.google.com/p/log4go/examples/XMLConfigurationExample.go:5: main redeclared in this block
    previous declaration at code.google.com/p/log4go/examples/SocketLogWriter_Manual.go:9
# github.com/ziutek/mymysql/examples
github.com/ziutek/mymysql/examples/prepared_stmt.go:10: printOK redeclared in this block
    previous declaration at github.com/ziutek/mymysql/examples/long_data.go:14
github.com/ziutek/mymysql/examples/prepared_stmt.go:14: checkError redeclared in this block
    previous declaration at github.com/ziutek/mymysql/examples/long_data.go:18
github.com/ziutek/mymysql/examples/prepared_stmt.go:26: main redeclared in this block
    previous declaration at github.com/ziutek/mymysql/examples/long_data.go:25
github.com/ziutek/mymysql/examples/reconnect.go:11: main redeclared in this block
    previous declaration at github.com/ziutek/mymysql/examples/prepared_stmt.go:26
github.com/ziutek/mymysql/examples/simple.go:10: printOK redeclared in this block
    previous declaration at github.com/ziutek/mymysql/examples/prepared_stmt.go:10
github.com/ziutek/mymysql/examples/simple.go:14: checkError redeclared in this block
    previous declaration at github.com/ziutek/mymysql/examples/prepared_stmt.go:14
github.com/ziutek/mymysql/examples/simple.go:22: checkedResult redeclared in this block
    previous declaration at github.com/ziutek/mymysql/examples/prepared_stmt.go:21
github.com/ziutek/mymysql/examples/simple.go:27: main redeclared in this block
    previous declaration at github.com/ziutek/mymysql/examples/reconnect.go:11
github.com/ziutek/mymysql/examples/transactions.go:11: printOK redeclared in this block
    previous declaration at github.com/ziutek/mymysql/examples/simple.go:10
github.com/ziutek/mymysql/examples/transactions.go:15: checkError redeclared in this block
    previous declaration at github.com/ziutek/mymysql/examples/simple.go:14
github.com/ziutek/mymysql/examples/transactions.go:15: too many errors
 !     Heroku push rejected, failed to compile Go app

[Dave Cheney]

The examples in some on the packages you are importing are not go get'able. If you have a look at some of the packages you will probably find they are single file examples intended to be run via go run. 

If that is the case the simplest example would be to copy those packages into your project so go get will skip over them. 

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Oliver Castle]

How come on my local machine I can use go get and reference them without a problems but when the same command is run on the Heroku build it is throwing the errors? That is the part that confused me?

[Dave Cheney]

I cannot explain that. I suggest setting GOPATH to a temporary location and repeating the steps that the heroku build pack is performing. 

[Carlos Castillo]

Whenever I upgrade to the latest version of tip, I run go install -v all, to re-build all the packages I have installed. Invariably, there are some external packages in my go path that have bad examples that are either out of date, or meant to be run on their own by go run. Unfortunately when you go get a package by name (or by import), it downloads the entire repo, but only builds the desired package, so this is often not a problem considered by many package developers. But the bad code is lying around to cause problems for a system which attempts to build all potential packages in the repositories it downloads.

I suspect this is what is going on with the heroku system. To be nicer to it's users, it downloads all packages and attempts to build every potential package in the repositories it downloads (go get ./...), but since that can easily lead to errors in packages that are pulled in as part of poorly designed repositories, the build can easily fail.

For example consider the following minimal example: http://play.golang.org/p/kwvIUhNfAH

If I toss that into its own package, lets call it foo, in its own GOPATH, so there is only one file at: $GOPATH/src/foo/main.go

The following commands on a completely bare GOPATH (only the above file) succeed without errors:

• go get -x foo
• cd $GOPATH/src/foo ; go get -x

The following, which suggested by the output is what heroku does instead, does not succeed because of the "broken" examples in log4go:

• cd $GOPATH/src ; go get -x ./...

I think documentation about how to create good examples in repositories, for example using a directory with a leading underscore to "hide" it from the go tool, might be a good start.

[Keith Rarick]

On Tue, Mar 19, 2013 at 3:05 PM, Carlos Castillo <cook...@gmail.com> wrote:
> I suspect this is what is going on with the heroku system.

We don't have to speculate. :)
The the Go Heroku buildpack is this shell script:
https://github.com/kr/heroku-buildpack-go/blob/master/bin/compile

> To be nicer to
> it's users, it downloads all packages and attempts to build every potential
> package in the repositories it downloads (go get ./...), but since that can
> easily lead to errors in packages that are pulled in as part of poorly
> designed repositories, the build can easily fail.

Note, this command tries to run in the root of the git repo for the app
being pushed to Heroku. That command should cause go to fetch
and install all packages contained in the Heroku app's repo, as
well as their explicitly named dependencies, but not necessarily
all packages contained in the repos of the dependencies.

> The following, which suggested by the output is what heroku does instead,
> does not succeed because of the "broken" examples in log4go:
>
> cd $GOPATH/src ; go get -x ./...

In this example, the equivalent of the buildpack's behavior is meant
to be:

cd $GOPATH/src/foo ; go get ./...

However, the buildpack relies on a file called .godir to know the
name of this directory. If that file contains something other than
the relative path from $GOPATH/src to the app's root, the buildpack
will do something wrong.

Ollie Castle: can you share the contents of file .godir in your repo?
Also, can you tell us the path on your local system where you're
successfully building the package?

I'd welcome any suggestions from the community on how better
to solve the problem of naming the package being built. The existing
design, .godir, has proven to be error-prone.

Some possible improvements:

- Sanity checking the contents of .godir. In addition to ensuring the
file exists, also make sure it's not empty, contains at least one
path component other than '.', etc.
- Have the buildpack print out the directory where it runs go get.

Other thoughts?

[Carlos Castillo]

go list could be useful here. At it's simplest level, go list ./... in $GOPATH/src will give a list of all potentially valid packages in that directory. This could be used at least as a hint in the error message to the user for valid values for the .godir file.

More advanced uses of the command could also have value. The following prints all package names, and the import paths. Filtering that looking for the package name "main", finds all packages that could yield an executable.

go list -f '{{ .Name }} : {{ .ImportPath }}' ./... | grep "^main " | awk -F " : " '{ print $2 }'

--
Carlos Castillo

[Ruslan Khusnullin]

Using Go-Buildpack isn't good idea at all. Too complicated. I use
Null-Buildpack instead:
http://github.com/ryandotsmith/null-buildpack.git

Pros:
- You can use tip compiler instead of version 1.0.3
- You manage your packages locally
- You compile locally in seconds and have chance to iterate
fix-compile-test very fast
- Your deployment is super fast because it pushes already compiled binary

Heroku uses 64-bit architecture while I'm stuck with 32-bit. That's
why my scripts are a bit complicated:

; cat build
#!/usr/bin/rc
GOARCH=386 go build -o httpd32 &&
CGO_ENABLED=0 GOARCH=amd64 go build -o httpd64

Note: I needed to do some work for enabling cross-compiling, search
golang.org for this if you're still on 32-bit too

; cat Procfile
web: ./run

; cat run
#!/bin/sh
case `uname -m` in
i?86)
exec ./httpd32
;;
x86_64)
exec ./httpd64
;;
*)
echo unknown machine hardware
exit 1
;;
esac

This `run` script keeps me happy on any machine I use.

Try it and you will love it.

[Johann Höchtl]

Am Mittwoch, 20. März 2013 13:50:51 UTC+1 schrieb Ruslan Khusnullin:

Using Go-Buildpack isn't good idea at all. Too complicated. I use
Null-Buildpack instead:
http://github.com/ryandotsmith/null-buildpack.git

Pros:
- You can use tip compiler instead of version 1.0.3

That last point has hit me as I ran into bugs in 1.0.3.

1. Does the buildpack https://github.com/kr/heroku-buildpack-go offer a way to fetch a newer version of Go?
2. With Go 1.1. round the corner (and the answer to 1. is no) --- is there a tentative update plan when 1.1. will be available on Heroku?
3. When using http://github.com/ryandotsmith/null-buildpack.git: Probably the best strategy is to create a  script which copies all necessary files (including binaries!) into a directory structure of its own and commit / push that to heroku?

Using https://github.com/kr/heroku-buildpack-go has the simplicity of adding only a a heroku repo to an existing eg. github repo but the downside of commiting eg. sql stored procedures source files / QML-files, which are unnecessary for program execution.

>
>

[Keith Rarick]

On Wed, Apr 10, 2013 at 12:42 AM, Johann Höchtl
<johann....@gmail.com> wrote:
> 1. Does the buildpack https://github.com/kr/heroku-buildpack-go offer a way
> to fetch a newer version of Go?

Yes.

There's a "go1.1" branch now, which you can use by setting
your buildpack url to
https://github.com/kr/heroku-buildpack-go.git#go1.1.

Also, you can get any version available on
https://code.google.com/p/go/downloads/list if you use
user-env-compile and set config var GOVERSION.
For example, if 1.1beta1 were still there for linux amd64,
you could do:

$ heroku labs:enable user-env-compile
$ heroku config:set GOVERSION=1.1beta1

See https://devcenter.heroku.com/articles/labs-user-env-compile.

> 2. With Go 1.1. round the corner (and the answer to 1. is no) --- is there a
> tentative update plan when 1.1. will be available on Heroku?

I intend to make 1.1 the default as soon as I know it's released.

[Johann Höchtl]

On 04/13/2013 08:57 AM, Keith Rarick wrote:
> On Wed, Apr 10, 2013 at 12:42 AM, Johann Höchtl
> <johann....@gmail.com> wrote:
>> 1. Does the buildpack https://github.com/kr/heroku-buildpack-go offer a way
>> to fetch a newer version of Go?
>
> Yes.
>
> There's a "go1.1" branch now, which you can use by setting
> your buildpack url to

> .
>
> Also, you can get any version available on
> https://code.google.com/p/go/downloads/list if you use
> user-env-compile and set config var GOVERSION.
> For example, if 1.1beta1 were still there for linux amd64,
> you could do:
>
> $ heroku labs:enable user-env-compile
> $ heroku config:set GOVERSION=1.1beta1
>
> See https://devcenter.heroku.com/articles/labs-user-env-compile.
>
>> 2. With Go 1.1. round the corner (and the answer to 1. is no) --- is there a
>> tentative update plan when 1.1. will be available on Heroku?
>
> I intend to make 1.1 the default as soon as I know it's released.
>

Those are all fascinating news. I tried it immediately, and in fact it
all was a matter of

heroku config:set
BUILDPACK_URL=https://github.com/kr/heroku-buildpack-go.git#go1.1

heroku push # Hm, is there a better way just to tell: rebuild!

heroku ps:scale <to whatever>

and now it's up and running. Wow!