Maven retrieved antlr4-4.5.3-sources.jar incomplete?

[Gerald Rosenberg]

Using Maven to retrieve the jar and corresponding source, based on:

        <dependency>
            <groupId>org.antlr</groupId>
            <artifactId>antlr4</artifactId>
            <version>4.5.3</version>
        </dependency>

The retrieved antlr4-4.5.3-sources.jar does not include the source for the runtime (v4 or v3) or ST4. 

For source lookup, Eclipse, at least, expects the contents of the source jar to correspond to the code jar. 

Is there a different artifact that I should be using?  Workaround?

Thanks,
Gerald

[Jim Idle]

       <dependency>
            <groupId>org.antlr</groupId>
            <artifactId>antlr4-runtime</artifactId>
            <version>4.5.3</version>
        </dependency>

You only need the runtime jar for generated code. As you are probably aware, for the ANTLR tool you need:

            <plugin>

                <groupId>org.antlr</groupId>
                <artifactId>antlr4-maven-plugin</artifactId>
                <version>4.5</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>antlr4</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

Jim

--
You received this message because you are subscribed to the Google Groups "antlr-discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to antlr-discussi...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

​

[Gerald Rosenberg]

My application requires both the tool and the runtime.  The source for the runtime is not present in the corresponding sources jar.  Eclipse expects a 1:1 match.

​

[Jim Idle]

Right - that's how it works.

That dependency pulls in the runtime code, and the source jar is available too. If you expand the maven dependencies and pick a .class file at random and double click it it will open the source for the class. You do need to tell Eclipse/Maven to download the source and docs jars otherwise you will have to right click on the dependency and tell it to do that.

Jim

On Fri, May 27, 2016 at 2:42 PM, Gerald Rosenberg <gbrose...@gmail.com> wrote:

Actually the issue is that I need the source for the runtime to be present in the sources jar.  For source lookup by the IDE.

[Gerald Rosenberg]

Yes, well understood and effectively what I am doing. But the runtime jar does not include the tool.  And, I need tool source lookup as well.

Having both the binary (and source) tool and runtiime jars in the project is problematic for several reasons.  Notably, the binary tool jar includes the runtime, meaning there is a redundant source of class files. Putting the runtime jar before the tools jar in the classpath is obvious. However, seems Eclipse source lookup can get confused as to the true source jar to use when doing the lookup from a text hyperlink. Happens most frequently when a tool source lookup has just been performed.  Could be source lookup is not using a pure classpath resolver.

The other is that the project will then ship dual runtimes.  Theoretically, not a problem, but not desirable as a practical matter.

Is there some particular reason why the tools source jar should not have a 1:1 correspondence with the contents of the tools binary jar, i.e., also include the runtime source?

Thanks,
Gerald

[Jim Idle]

Let's step back here a minute. Why do you need the tool source code in your Parser project. You don't normally but you may well have good reason. Eclipse and maven are not to hot on tracing into tools encapsulated in plugins, which is feuding when you are trying to develop compilers. 

On Fri, May 27, 2016 at 10:06 AM -0700, "Gerald Rosenberg" <gbrose...@gmail.com> wrote:

Yes, well understood and effectively what I am doing. But the runtime jar does not include the tool.  And, I need tool source lookup as well.

Having both the binary (and source) tool and runtiime jars in the project is problematic for several reasons.  Notably, the binary tool jar includes the runtime, meaning there is a redundant source of class files.  Seems Eclipse source lookup can get confused as to the true source jar to use when doing the lookup from a text hyperlink. Happens most frequently when a tool source lookup has just been performed.  Could be source lookup is not using a pure classpath resolver.

The other is now my project is setup to ship dual runtimes.  Theoretically, not a problem, but not desirable as a practical matter.

Is there some particular reason why the tools source jar should not have a 1:1 correspondence with the contents of the tools binary jar, i.e., also include the runtime source?

Thanks,
Gerald

--

[Gerald Rosenberg]

An example application is AntlrDT, an Eclipse plugin source code editor and builder for Antlr grammars.  The Antlr tool is required to build the source in real time order to generate all errors -- simply parsing the source only recognizes basic syntax errors.  Tool is also used to build the grammar being edited and parse input appropriate for that grammar in order to execute unit tests (using SnippetTest - github.com/grosenberg/SnippetsTest) and to build a graphical representation of the corresponding parse tree. 

Another is XVisitor (github.com/grosenberg/xvisitor) - generates parse tree visitors from Antlr-like symbolic grammars.  Like Antlr, it is both a generator and runtime.  The generator requires both the Antlr tool and runtime.

Not the most common of uses, but not particularly outside the ordinary course.

FWIW, I have not had any trouble debugging through jars in general. To be certain, I am mostly interested in simple source lookup directly from the editor - the projects keep getting bigger and my memory smaller. ;)

[Jim Idle]

Ok. Gotya :)

While the sources are there:

http://repo1.maven.org/maven2/org/antlr/antlr4/4.5.3/

They should not be including the runtime. Maybe this the complete jar. Take a look at what the maven plugin Pom uses. That's how you want to do it. However it's possible that the tool stuff isn't quite kosher as the maven build is now a poor cousin of the python based build that Ter uses ;) it may be the case that a standalone tool artifact needs to be published. 

I would delve deeper but I am on a beach until Thursday :)

Jim

[Eric Vergnaud]

Jim, FYI we no longer rely on bild.py for building the tool and java runtime. It is now all maven based.

[Jim Idle]

I must have missed that Nemo Etic! ;)

[Gerald Rosenberg]

@Jim: The maven plugin uses an antlr4-master artifact that does not appear to be in the public repository - actually references a 4.5.4 snapshot, but Eclipse is currently unable to find a corresponding 4.5.3 master.

Thanks and enjoy the sun.

@Eric: the POMs for these builds are not on GitHub.  Are they visible elsewhere?  Would be happy to provide a pull-request to fix, given a dev decision as to the desired content of the antlr4 artifact binary jar.

[Eric Vergnaud]

???

what about https://github.com/antlr/antlr4/blob/master/pom.xml ?

[Gerald Rosenberg]

Are you suggesting that one POM produces the three different jar artifacts, antlr4-master, antlr4, and antlr4-runtime and their corresponding source jars? 

Admittedly, my knowledge of Maven is limited, but it looks to produce an all-in-on jar -- including the maven tool and test suites. Not clearly one of the maven artifacts.  And, in any case, not really appropriate to ship as part of a released applications of the type described above.

The github POM has a parent pom not visible, so I am unable to see an indication of all of the other module POMS that exist.  Since it is typical to use multiple POMS to generate content variant jars from a codebase, I presumed that the devs were doing something like that, and perhaps also automating the uploading of the three distinct artifacts to Maven central.

If that is not what is being done, do you know what procedure is being used to cut specifically the antlr4 artifact binary and source jars.

[Eric Vergnaud]

I'm not suggesting, I'm telling.

The master pom triggers the build of the module poms located in module directories.

So the way to build antlr is to move to the root dir and type mvn. 

Maybe you should try it to understand what it does.

[Gerald Rosenberg]

Look, the POM builds a number of snapshot binary jars. It does not define or build the source jars - does not even include the source or release plugins.

Not unreasonable to believe that something else is involved in the process of cutting releases including, specifically, the generation of the antlr4 artifact. That further stands to reason due to the inconsistency in the contents of the antlr4 artifact binary and source jars.

And yes, that was the entire point of this thread - how to account for the inconsistency in the contents of the antlr4 artifact binary and source jars.

Next time, try exercising reading comprehension instead of condescension.

[Eric Vergnaud]

Sorry if I sounded condescendant, definitely not the intent.

I was just being assertive, since you wrote "my knowledge of Maven is limited"

You also wrote earlier "the POMs for these builds are not on GitHub" when they are, which didn't help.

The sources-jar are generated as part of the OSGI release, which I believe occurs when you do mvn publish

To get the same result locally, you may also want to try mvn source:jar

Overall you are looking for help on maven, not on ANTLR so this may not be the right forum.

The above is much better documented on the maven web site than any help you can get in this forum.

[Gerald Rosenberg]

Eric, just stop.

You seem incapable of understanding that the issue involves the antlr4 artifact. There is a discrepancy in the artifact as published to Maven (central or whatever).

Suggesting that this is a general Maven issue I can solve on my own is just BS.  And, FWIW, my acknowledging that I don't know everything is no just cause for you to be 'assertive', though that is not the word I would use.

[Jim Idle]

Eric,

I'm not sure if we are publishing the artifacts that are needed here. We should be unless the antlr plugin is being built in a strange way - I could not find the standalone tool artifact in maven central and it makes me wonder. With just a phone at my disposal I am not able to look at the source very easily. 

Gerald, in the short term you may be able to use the source code to build the tool artifact in to your local repo, which gives you the singular correspondence that you will need for eclipse etc

[Gerald Rosenberg]

@Jim - no problem with me doing it for myself.  Just wanted to give back to the larger community and help identify and fix a problem.

There is no rush for this, so enjoy your vacation.

[Eric Vergnaud]

Jim,

There is no "standalone tool jar", simply the antlr4 jar (the project is called "tool", but the artifactId is antlr4).

The runtime and ST4 are dependencies of the tool, not part of the tool. So is abego.treelayout.

As a convenience, they are packaged with the tool using the maven shade plugin, this avoids adding their jars to the class path.

The maven shade plugin is a very standard plugin, I don't see anything "strange" here.

To be honest, I have never seen any project publishing the source code of their dependencies.

As such, I believe there is no discrepancy or inconsistency in the published jars:

tool/antlr4:

http://search.maven.org/remotecontent?filepath=org/antlr/antlr4/4.5.3/antlr4-4.5.3.jar

with its source here:

http://search.maven.org/remotecontent?filepath=org/antlr/antlr4/4.5.3/antlr4-4.5.3-sources.jar

runtime:

http://search.maven.org/remotecontent?filepath=org/antlr/antlr4-runtime/4.5.3/antlr4-runtime-4.5.3.jar

with its source here:

http://search.maven.org/remotecontent?filepath=org/antlr/antlr4-runtime/4.5.3/antlr4-runtime-4.5.3-sources.jar

Anyone wanting to display source code for a dependency packaged with antlr4 should follow the strategy you described earlier.

Eric

n.b. thanks for providing patient "beach" support on this :-), sometimes I just wonder why I provide free code and free support to people who believe their expectation is the spec...

[Gerald Rosenberg]

Amazing. I post an observation and question regarding an apparent inconsistency, and further detail the circumstances where it creates a problem. 

Jim, on understanding the issue, acknowledged that the antlr4 artifact should probably not include the runtime code. I offered to do the work to provide a fix once the devs decide the correct outcome.

Amazing that you would then get self-righteously indignant and accuse me of demanding a change.

Not so amazing that others do not regard you with the same level of respect that you alone believe you are entitled to.

Yes, you have contributed.  I have as well.  Others can value the relative merits if they so choose. But, no one would endorse your trying to shut down discussion that does not strictly comport with your expectations.

[Jim Idle]

Ah. We should be publishing the tool without the shaded version. We can publish the shaded version as well but the shaded version is fit the command line really.