How to build with maven?

[Denis Haskin]

In https://github.com/openplans/OpenTripPlanner/issues/1022 Andrew comments that the GettingStartedMaven wiki page isn't correct...

So what is the right incantation to build via maven?  0.9.1 and 0.9.0 both fail to build for me (OSX or Ubuntu).  If I try something like

    mvn install  -DskipTests -P interactive

It eventually tries to run

    /bin/sh -c cd <otphome>/opentripplanner-integration && java -Xmx900m -classpath <reallylongclasspath> org.opentripplanner.integration.graph_builder.GraphBuilderIntegrationMain

and fails because GraphBuilderIntegrationMain is looking for "src/main/resources/graph-builder.xml" relative to the OTP home directory, not relative to <otphome>/opentripplannerintegration.  The cd <dir> && <command> idiom doesn't seem to be doing what's expected (perhaps this is a maven problem?).

(Apologies for repeating here what I put in as a comment on #1022)

--

Denis Haskin

cell: 781-258-7414

[Andrew Byrd]

Hi Denis,

I am able to reproduce this problem locally even with the current master
branch. It looks like the failure is due to a missing GTFS input file
that was hosted on a server that has been shut down. I'll try to find
another copy to post elsewhere.

In the mean time, anyone who wants to run a build (current or older
releases) can simply remove the integration-testing submodule. Just edit
the top-level pom.xml and remove the line:
<module>opentripplanner-integration</module>

The build can then be performed in the top level of the OTP repo with:
mvn clean package -DskipTests

Leave the -DskipTests off if you don't mind a somewhat slower build.
Make sure you're using a recent version of Maven. Some problems with
multi-module builds were recently resolved. I'm on 3.0.4 and no longer
have these problems.

This reveals another negative side of including the integration tests in
the Maven build -- builds of older releases depend on hard-coded links
to files that may not be hosted in the same place forever. Running the
server through a Maven plugin buries the real error messages several
layers down and it's not at all obvious what's going on.

I have just come up with a good alternative to the current approach of
using Maven plugins to download servlet containers and run them with
shell commands (shudder) and we are well on the way to removing or
heavily altering this module... more on that later.

-Andrew

> --
> You received this message because you are subscribed to the Google
> Groups "OpenTripPlanner Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to opentripplanner...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Denis Haskin]

Thanks -- yeah, maven is really good at hiding the actual error from you.  Seems to have been engineered specifically to do that.

In any event, I found where the GTFS file is referenced and have swapped in another one (trimet!) and so trying that now.

What we found on OBA is that integration tests had to rely only on resources that were bundled with the project.  A pain, especially with resources like GTFS files.

One thing to maybe consider for servlet containers is to use jetty; there's a decent maven jetty plugin and so maven handles downloading it for you.  Worked pretty well for us on OBA (well, hat tip to Brian who set it up like that in the first place).

(Yeah, the tomcat distribution it's trying to get to seems to no longer be there, either: http://www.apache.org/dist/tomcat/tomcat-6/v6.0.35/bin/apache-tomcat-6.0.35.zip .  I can work around that.)

--

Denis Haskin

cell: 781-258-7414

[Andrew Byrd]

On 05/15/2013 10:34 PM, Denis Haskin wrote:
> Thanks -- yeah, maven is really good at hiding the actual error from
> you. Seems to have been engineered specifically to do that.

It should be less frustrating once we eliminate all the clever hacks
from the project configuration and use it only for dependency
management, compilation, and packaging in a completely standard way.

> In any event, I found where the GTFS file is referenced and have swapped
> in another one (trimet!) and so trying that now.

Great, let me know if you run into any problems. And don't hesitate to
work with the master branch, it should certainly be stable enough for
experimentation.

> What we found on OBA is that integration tests had to rely only on
> resources that were bundled with the project. A pain, especially with
> resources like GTFS files.

We're pulling a stress testing / profiling module completely out of the
Maven build (~2 hours of pseudorandom requests against each
commit). This way we can make the build less brittle by including all
the (relatively smaller) unit test resources/fixtures in the repo, still
getting constant quality feedback.

> One thing to maybe consider for servlet containers is to use jetty;
> there's a decent maven jetty plugin and so maven handles downloading it
> for you. Worked pretty well for us on OBA (well, hat tip to Brian who
> set it up like that in the first place).

I was just working through this yesterday. In my opinion, deploying a
simple REST interface like OTP's into an external Servlet/JSP container
like Tomcat seems like overkill, let alone "Enterprise" application
servers like Glassfish.

I had already started along the path of embedding Jetty into OTP when I
realized there was another option: Grizzly, which is the HTTP
abstraction layer of Glassfish, can interface directly with Jersey. The
resulting class is nice and succinct:

https://github.com/openplans/OpenTripPlanner/blob/master/opentripplanner-api-webapp/src/main/java/org/opentripplanner/api/servlet/GrizzlyServer.java

From my perspective, this is what the future of OTP looks like: self
contained and lightweight.

-Andrew

[Andrew Byrd]

In the interest of not breaking the build for old tags, I placed a GTFS
file at the URL expected by the integration test module. With that
problem resolved, as Denis reported the cargo plugin now cannot find
Tomcat at the expected address.

As I am working on a self-contained OTP module that does not need a
servlet container, the time has come to remove this step from the build.
Let's also take this as a hint that in the future we should entirely
avoid clever tricks and pseudo-imperative style in the Maven config,
opting for just dependency management, compilation, and packaging
according to convention.

The integration testing phases happen between the package and install
phases, so if you want to just _package_ the artifacts for an older tag
of OTP rather than _install_ them you don't need to remove the
integration testing module. Just do:
mvn clean package [-DskipTests]

If on the other hand you want the build to reach _install_ or later
phases, you'll want to remove the integration testing module:

Just edit the top-level pom.xml and remove the line:
<module>opentripplanner-integration</module>

And do:
mvn clean package [-DskipTests]

I have removed the cargo invocation from the integration module's POM,
so the above work-arounds will only be necessary when building past commits.

-Andrew

[Andrew Byrd]

On 05/17/2013 03:57 AM, Andrew Byrd wrote:
> If on the other hand you want the build to reach _install_ or later
> phases, you'll want to remove the integration testing module:
> Just edit the top-level pom.xml and remove the line:
> <module>opentripplanner-integration</module>
>
> And do:
> mvn clean package [-DskipTests]

Correction: that should be
mvn clean install [-DskipTests]