Antlr4-master build issues

[Graham Wideman]

Hi (probably for Sam?):

In hopes of using the latest runtime code that you fixed just for me :-), I downloaded antlr4 based on today's commit 81c4ebbf6f, and tried to build both the runtime and tool (in NB 7.2 using maven etc).  I thought I should use latest Tool in case it generates different code to use with most recent runtime.

Hit a couple of issues while building:

Runtime: the pom has a dependency on ST 4.0.4 which doesn't seem to be needed (but attempts to drag in other dependencies). So I deleted it and build proceeds OK.

Tool: Has a dependency on antlr-runtime-3.4.1-SNAPSHOT.jar. 

How do I resolve this? Maven tries to fetch it from http://antlr.org/antlr-snapshot/org/antlr/antlr-runtime/3.4.1-SNAPSHOT/antlr-runtime-3.4.1-SNAPSHOT.pom, which fails.
I considered downloading some antlr 3.4 or 3.5 jar and fudging the dependency to point to that, but didn't want to proceed just randomly.

Suggestions?  Thanks!

-- Graham

[Sam Harwell]

The Tool requires a build of ANTLR 3.4 build with generics support (target 1.5 or newer). All of the builds currently available through Maven central use target 1.4. Until this is resolved, you’ll need to build the latest ANTLR 3 runtime from source before building v4.

 

The tool definitely depends on ST4. The latest version is 4.0.6, but 4.0.4 is the latest in Maven central.

 

--

Sam Harwell

Owner, Lead Developer

http://tunnelvisionlabs.com

--
 
 

[Graham Wideman]

Thanks, Sam. Couple of points:

> The tool definitely depends on ST4. The latest version is 4.0.6, but 4.0.4 is the latest in Maven central.

Yes, but so far as I can tell, runtime does not depend on ST4, yet runtime's pom says it does.

> The Tool requires a build of ANTLR 3.4 build with generics support (target 1.5 or newer).

I take it you're talking about Java 1.5 (or 6 or 7) vs Java 1.4, right?

> Until this is resolved, you’ll need to build the latest ANTLR 3 runtime from source before building v4.

And again to avoid heading into ditch, by this you mean: grab latest source from https://github.com/antlr/antlr3, right?

Minor confusion on that page: the README.txt (visible immediately) says it's version 3.5. However, on checking the overall pom, and the pom for the Java source,they say 3.4.1-SNAPSHOT, so that looks like the right thing.

OK, I'll give that a try.  Thanks,

-- Graham

[Sam Harwell]

You’re on the right track.

 

We should have dependencies and other issues straightened out next month for a coordinated release of the following, including proper publishing at Maven central.

 

·         ANTLR 3.5.1 (update to address publishing issues)

·         StringTemplate 4.0.7 (update to address publishing issues)

·         ANTLR 4.0!

--
 
 

[Graham Wideman]

FWIW, I succeeded, but not easy going.  In case anyone else is going to try this:

I used NetBeans 7.2, though most of the job is carried out by maven (included with NB, maven 3.0.4). NB helps by showing syntax errors for things that haven't been resolved, and for navigating the source more easily.

The job consists of

• downloading the master source for both antlr 3.4.1 and antlr 4.latest.
• First build 3.4.1 runtime.
  
• I found I was able to do this by using NB > Open Project, and browsing ...\runtime\Java. NB opened this as a project. I ran Build, which fired off maven to download all sorts, compile etc, and this produced antlr-runtime-3.4.1-SNAPSHOT.jar (and corresponding source jar).
• Now build 4.0 runtime.  I used pretty much the same procedure, and I think this went OK. After this
  
• I copied 4.0 runtime jar and source jar to my generic 3rd-party javalibs directory, and created an ANT library reference for it, for use with my ANTLR-using projects.
  
• However, now projects built with parser code generated with antlr4-b3 didn't compile with the new 4.0 runtime, because some of the method signatures have changed. So I needed the latest Tool as well.
• Build 4.0 Tool. This is the stinky part.
• Attempting to build resulted in various "can't find dependency" errors.
  
• In theory you should be able to get dependencies on 3.4.1 runtime and 4.0 runtime to resolve by pointing the dependencies to the above-mentioned projects, however this didn't seem to work. This may be partly because when these projects build and are placed into the maven repo, they are accompanied by poms that inherit from a parent pom, that's not in the repo. Consequently maven can't find the complete metadata for the jars in the repo. Or something like that.
• My solution was to:
• Copy the 3.4.1 runtime and 4.0 runtime jars to my javalibs directory
• Delete them from the maven repo (along with their metadata files)
• Hide the corresponding projects from NB and maven
• Create new dependencies to the moved jars.
• Use NB "manually install artifact" to install the two just-moved jars into the repo, now minus their poms.
• You might be able to get away with just deleting the pom files from the repo, I'm not sure.
  
• Then I could get 4.0 Tool to build.
• It failed one of the tests, so maven didn't produce a jar. That can be surmounted by skipping the tests; Tools > Options > Java > Maven > "Skip Tests for..." checkbox.
  
•  I then moved the 4.0 Tool jar to my javalibs directory, along with ST4-4.0.4.jar (which got fetched into the local repo at some point).
• Added runtime 4, tool 4, runtime 3.4.1 and ST4 jars to CLASSPATH.
• Voila, working Tool 4.

Other hints:

• I had to experiment a lot with the dependencies, and when creating each one it's necessary to know the maven "coordinates" (groupId, artifactId, version) for the dependency you're trying to create. So I found it essential to have a pristine copy of the source tree for the project so I could easily take a look at the original state of the relevant pom file.
• At some point, from the antlr 4 master pom.xml, I deleted the repositories block, to stop maven from erroring on attempts to look for non-existent antlr.org/antlr-snapshot.
  

OK, that's not sufficiently detailed to do it by rote, but hopefully enough clues that it saves others some time.

-- Graham

[Sam Harwell]

Thanks Graham. Hopefully this helps anyone who’s trying to work with this project right now.

 

I’ve applied for hosting the Maven artifacts at Sonatype. Hopefully I can get the updated V3 runtime hosted in Maven central soon and end this madness. Once those binaries are online, building V4 will be as simple as building the master POM in the v4 repo’s root folder.

 

Thanks,

--

Sam Harwell

Owner, Lead Developer

http://tunnelvisionlabs.com

 

From: antlr-di...@googlegroups.com [mailto:antlr-di...@googlegroups.com] On Behalf Of Graham Wideman
Sent: Monday, November 26, 2012 10:31 PM
To: antlr-di...@googlegroups.com
Subject: Re: [antlr-discussion] Antlr4-master build issues

 

FWIW, I succeeded, but not easy going.  In case anyone else is going to try this:

--
 
 

[Graham Wideman]

I'm looking forward to all this being on Google Play store, where we can just search for the project we were thinking of building, and download it already completed. :-)

[Graham Wideman]

Revised build steps for antlr4-latest.

In an earlier email, I posted some convoluted steps for persuading NetBeans-with-maven to build antlr-3.4.1 and ultimately antlr-4.  I have since revisited this from scratch, and found the process can be much simpler, no editing of pom files required. The key is to NOT try to build the sub-projects for runtime and tool separately, but rather to build the over-arching projects, so that all the needed parts, including parent poms, wind up in the maven repo for subsequent steps.

FWIW, I started by deleting all previously generated jars, including the org/antlr branch of the local maven repo ([userdir]/.m2/repository).

• Download source zips for antlr 4.latest, and antlr 3.4.1-SNAPSHOT.
  
• Unzip and arrange so you have folders like the following:
• somedir
• antlr3-master
• antlr4-master.
• Do not revise contained folders, as these are organized in accordance with maven principles, and coordinate with provided pom.xml files.
• Building 3.4.1-SNAPSHOT
• In NetBeans, File > Open Project and in the  Open Project dialog, navigate to antlr3-master. This folder should appear with an "Ma" icon, indicating a maven project. However, the icon may not appear immediately, which previously led me to think this wasn't a viable project. But it is, so select it.  At that point the dialog should offer the "Open Required Projects" checkbox, check it. Hit Open Project.
• This adds the following projects to the NB project tree:
• ANTLR3 Maven Archetype
• ANTLR gUnit
• ANTLR Master build control POM   <----
• ANTLR Runtime 3.4.1-SNAPSHOT
• ANTLR Tool 3.4.1-SNAPSHOT
• Maven plugin for ANTLR V3
• Maven plugin for gUnit ANTLR V3
• Select "ANTLR Master build control POM", and choose Build.  This process downloads some needed dependencies, so obviously your computer has to be online for this. This gave me the following partial success:
  

------------------------------------------------------------------------
Reactor Summary:
ANTLR Master build control POM .................... SUCCESS [0.328s]
Antlr Runtime 3.4.1-SNAPSHOT ...................... SUCCESS [2.527s]
ANTLR Tool 3.4.1-SNAPSHOT ......................... SUCCESS [9.313s]
Maven plugin for ANTLR V3 ......................... SUCCESS [1.077s]
ANTLR gUnit ....................................... FAILURE [2.932s]
Maven plugin for gUnit ANTLR V3 ................... SKIPPED
ANTLR3 Maven Archetype ............................ SKIPPED
------------------------------------------------------------------------
... but that's OK, since we are mostly interested in getting Runtime 3.4.1 and also the parent POM into the local maven repo for use in subsequent steps.

• Building 4.0-SNAPSHOT
• In NetBeans, File > Open Project and in the  Open Project dialog, navigate to antlr4-master. This folder should appear with an "Ma" icon, and like before you might have to wait for that.  Select it and hopefully the dialog offers "Open Required Projects" checkbox, check it. Hit Open Project.
• This adds the following:
• ANTLR 4 Maven plugin
• ANTLR 4                 <----
• ANTLR 4 Runtime
• ANTLR 4 Tool
• Before proceeding, you may want to select "ANTLR 4", and choose Set Configuration > release-profile, as this will generate source and javadoc jars along with the executable jar.
• Select "ANTLR 4" and choose Build.  After some grinding, you should find jars in the target directories for tool and runtime.

------------------------------------------------------------------------
Reactor Summary:
ANTLR 4 ........................................... SUCCESS [0.156s]
ANTLR 4 Runtime ................................... SUCCESS [2.481s]
ANTLR 4 Tool ...................................... SUCCESS [5.662s]
ANTLR 4 Maven plugin .............................. SUCCESS [1.255s]
------------------------------------------------------------------------

Other notes:

• This produces and/or fetches all the necessary jars for working with latest ANTLR4. However:
• It does not package them into an all-in-one jar like the stable release. Not sure how to do that.
• It does not install the jars anywhere or set CLASSPATH. You can manually copy them from the target dirs or repo to suit.

Hope that helps,

Graham

[Jim Idle]

Graham

You should find that there is a file BUILD.txt that explains the build. It will probably help. Just adding this here so that others see it. 

Jim

--
 
 

[Graham Wideman]

Hi Jim:

Yes, sorry, I forgot to mention that there's a BUILD.txt included in the ANTLR 3 zip, which is extensive and interesting (and written by Jim :-) ).   And, if I had read it thoroughly, I would have noticed that it points out the problem with trying to build the individual sub-modules -- maven complains of not finding the parent pom.  Unfortunately, the error message that maven produces is so obtuse that I didn't recognize that was what it was complaining about.

There's no build instructions included with ANTLR 4 that I encountered, but no complaint about that -- after all, it's still beta. And in theory it's straightforward.  :-)

-- Graham

[Sam Harwell]

I uploaded snapshot releases to sonotype, so they should be available for your Maven builds without requiring you to build them from source. The available releases are:

 

ANTLR 3: 3.5.1-SNAPSHOT

StringTemplate 4: 4.0.7-SNAPSHOT

ANTLR 4: 4.0-SNAPSHOT

 

--

Sam Harwell

Owner, Lead Developer

http://tunnelvisionlabs.com

 

From: antlr-di...@googlegroups.com [mailto:antlr-di...@googlegroups.com] On Behalf Of Graham Wideman
Sent: Tuesday, November 27, 2012 8:49 PM
To: antlr-di...@googlegroups.com
Subject: Re: [antlr-discussion] Antlr4-master build issues

 

Hi Jim:

--
 
 

[Graham Wideman]

Excellent -- thanks!
-- Graham

[Sam Harwell]

No problem. Please let me know if it works for you :)

 

--

Sam Harwell

Owner, Lead Developer

http://tunnelvisionlabs.com

 

From: antlr-di...@googlegroups.com [mailto:antlr-di...@googlegroups.com] On Behalf Of Graham Wideman
Sent: Tuesday, November 27, 2012 11:43 PM
To: antlr-di...@googlegroups.com
Subject: Re: [antlr-discussion] Antlr4-master build issues

 

Excellent -- thanks!
-- Graham

--
 
 

[Hans Uhlig]

Any chance you can upload the snapshot for the maven build plugin as well?

[Sam Harwell]

I uploaded all artifacts associated with the ANTLR 3 and ANTLR 4 projects, which includes gUnit for v3, and the maven plugins for v3 and v4 (haven’t tested the v4 plugin).

 

--

Sam Harwell

Owner, Lead Developer

http://tunnelvisionlabs.com

 

From: antlr-di...@googlegroups.com [mailto:antlr-di...@googlegroups.com] On Behalf Of Hans Uhlig
Sent: Thursday, November 29, 2012 9:55 PM
To: antlr-di...@googlegroups.com
Subject: Re: [antlr-discussion] Antlr4-master build issues

 

Any chance you can upload the snapshot for the maven build plugin as well?

 

[Hans Uhlig]

Thanks. I must be being dense as I cant find them. Am I looking in the wrong sonatype repo?

http://mvnrepository.com/artifact/org.antlr

https://repository.sonatype.org/index.html#nexus-search;quick~antlr4-runtime

[Sam Harwell]

I thought Maven would be able to find and automatically download them as part of a build. Is that not the case?

 

Currently they are in the snapshot repository:

https://oss.sonatype.org/content/repositories/snapshots/org/antlr/

 

--

Sam Harwell

Owner, Lead Developer

http://tunnelvisionlabs.com

 

From: antlr-di...@googlegroups.com [mailto:antlr-di...@googlegroups.com] On Behalf Of Hans Uhlig
Sent: Thursday, November 29, 2012 10:27 PM
To: antlr-di...@googlegroups.com
Subject: Re: [antlr-discussion] Antlr4-master build issues

 

Thanks. I must be being dense as I cant find them. Am I looking in the wrong sonatype repo?

 

http://mvnrepository.com/artifact/org.antlr

 

https://repository.sonatype.org/index.html#nexus-search;quick~antlr4-runtime

On Thursday, November 29, 2012 8:18:20 PM UTC-8, Sam Harwell wrote:

I uploaded all artifacts associated with the ANTLR 3 and ANTLR 4 projects, which includes gUnit for v3, and the maven plugins for v3 and v4 (haven’t tested the v4 plugin).

 

--

Sam Harwell

Owner, Lead Developer

http://tunnelvisionlabs.com

 

From: antlr-di...@googlegroups.com [mailto:antlr-di...@googlegroups.com] On Behalf Of Hans Uhlig
Sent: Thursday, November 29, 2012 9:55 PM
To: antlr-di...@googlegroups.com
Subject: Re: [antlr-discussion] Antlr4-master build issues

 

Any chance you can upload the snapshot for the maven build plugin as well?

 

--
 
 

[Hans Uhlig]

My fresh copy of Apache Maven seems to not like it.

I added the following and it found it but mine doesnt seem to find the oss snapshots repo

    <repositories>

        <repository>

            <id>sonatype-snapshots</id>

            <name>sonatype-snapshots</name>

            <url>https://oss.sonatype.org/content/repositories/snapshots</url>

            <snapshots>

                <enabled>true</enabled>

                <updatePolicy>always</updatePolicy>

            </snapshots>

        </repository>

    </repositories>

    <pluginRepositories>

        <pluginRepository>

            <id>sonatype-snapshots</id>

            <name>sonatype-snapshots</name>

            <url>https://oss.sonatype.org/content/repositories/snapshots/</url>

            <snapshots>

                <enabled>true</enabled>

                <updatePolicy>always</updatePolicy>

            </snapshots>

        </pluginRepository>

    </pluginRepositories>

[Sam Harwell]

Thanks for the info. :)

 

When ANTLR 4.0 is released, these artifacts will become available through Maven central so you won’t have to resort to these workarounds.

 

--

Sam Harwell

Owner, Lead Developer

http://tunnelvisionlabs.com

 

From: antlr-di...@googlegroups.com [mailto:antlr-di...@googlegroups.com] On Behalf Of Hans Uhlig

Sent: Thursday, November 29, 2012 11:33 PM
To: antlr-di...@googlegroups.com

--