Error when executing pattern from standalone Java

[IBWT]

Hello.

I have created the following patterns for the JDTAST metamodel of the Grabats 2009 contest which I think you may already know:

package grabats

import "org.amma.dsl.jdt.dom"
import "org.amma.dsl.jdt.primitiveTypes"
pattern isStatic(M : Modifier)= {
    Modifier.static(M, true);
}

pattern isPublic(M : Modifier)= {
    Modifier.public(M, true);
}

pattern methodOfType(T : TypeDeclaration, M : MethodDeclaration)= {
    TypeDeclaration.bodyDeclarations(T, M);
}

pattern methodWithReturnType(M : MethodDeclaration, S : String)= {
    MethodDeclaration.returnType(M, R);
    SimpleType(R);
    SimpleType.name.fullyQualifiedName(R, S2);
    check(S2.equals(S));
}

pattern grabats(T : TypeDeclaration, M : MethodDeclaration)= {
    find methodOfType(T, M);
    TypeDeclaration.name.fullyQualifiedName(T, S);
    find methodWithReturnType(M, S);
    MethodDeclaration.modifiers(M, MF1);
    MethodDeclaration.modifiers(M, MF2);
    MF1 != MF2 ;
    find isPublic(MF1);
    find isStatic(MF2);
}

I have tested the grabats pattern using the Query Explorer with a reduced model and it works fine. However, since the models I want to query are too big to be opened by the Ecore tree viewer and then queried by the Query Explorer, I have developed a standalone Java application that calls the generated pattern matcher. The code is something like this:

        ResourceSet rs = new ResourceSetImpl();
            rs.getResourceFactoryRegistry().getExtensionToFactoryMap()
                    .put("xmi", new XMIResourceFactoryImpl());
        EPackage.Registry.INSTANCE
                .put(DOMPackage.eNS_URI, DOMPackage.eINSTANCE);
        EPackage.Registry.INSTANCE.put(CorePackage.eNS_URI,
                CorePackage.eINSTANCE);
        EPackage.Registry.INSTANCE.put(PrimitiveTypesPackage.eNS_URI,
                PrimitiveTypesPackage.eINSTANCE);

        long l = System.currentTimeMillis();
       
        Resource r = rs
                .getResource(URI.createFileURI(new File(sourceFile)
                        .getAbsolutePath()), true);
       
        System.out.println("LOAD TIME: " + (System.currentTimeMillis() - l));
       
        l = System.currentTimeMillis();
       
        GrabatsMatcher matcher = new GrabatsMatcher(r);
        Collection<GrabatsMatch> matches = matcher.getAllMatches();
        for (GrabatsMatch match : matches) {
            System.out.println(match);
        }

        System.out.println("QUERY TIME: " + (System.currentTimeMillis() - l));

However, when I run this program I get the following exception:

1    [main] ERROR abats.grabats.GrabatsMatcherFactory  - Exception during parsePattern!
java.lang.NullPointerException
    at org.eclipse.viatra2.emf.incquery.runtime.util.XmiModelUtil.getGlobalXmiResource(XmiModelUtil.java:33)
    at org.eclipse.viatra2.emf.incquery.runtime.api.impl.BaseGeneratedMatcherFactory.getModelRoot(BaseGeneratedMatcherFactory.java:96)
    at org.eclipse.viatra2.emf.incquery.runtime.api.impl.BaseGeneratedMatcherFactory.parsePattern(BaseGeneratedMatcherFactory.java:62)
    at org.eclipse.viatra2.emf.incquery.runtime.api.impl.BaseGeneratedMatcherFactory.getPattern(BaseGeneratedMatcherFactory.java:44)
    at org.eclipse.viatra2.emf.incquery.runtime.api.impl.BaseGeneratedMatcher.<init>(BaseGeneratedMatcher.java:37)
    at grabats.grabats.GrabatsMatcher.<init>(GrabatsMatcher.java:64)
    at grabats.grabats.GrabatsMatcher.<init>(GrabatsMatcher.java:52)
    at es.modelum.test.Test.testGrabatsXMIIncQuery(Test.java:938)
    at es.modelum.test.Test.main(Test.java:192)
Exception in thread "main" java.lang.NullPointerException
    at org.eclipse.viatra2.emf.incquery.runtime.internal.matcherbuilder.EPMBuilder.getPosMapping(EPMBuilder.java:78)
    at org.eclipse.viatra2.emf.incquery.runtime.internal.matcherbuilder.EPMBuilder.getPosMapping(EPMBuilder.java:1)
    at org.eclipse.viatra2.gtasm.patternmatcher.incremental.rete.boundary.ReteBoundary.createProductionInternal(ReteBoundary.java:505)
    at org.eclipse.viatra2.gtasm.patternmatcher.incremental.rete.construction.ReteContainerBuildable.patternCollector(ReteContainerBuildable.java:277)
    at org.eclipse.viatra2.gtasm.patternmatcher.incremental.rete.construction.ReteContainerBuildable.patternCollector(ReteContainerBuildable.java:1)
    at org.eclipse.viatra2.emf.incquery.runtime.internal.matcherbuilder.EPMBuildScaffold.construct(EPMBuildScaffold.java:47)
    at org.eclipse.viatra2.emf.incquery.runtime.internal.matcherbuilder.EPMBuilder.construct(EPMBuilder.java:69)
    at org.eclipse.viatra2.emf.incquery.runtime.internal.matcherbuilder.EPMBuilder.construct(EPMBuilder.java:1)
    at org.eclipse.viatra2.gtasm.patternmatcher.incremental.rete.boundary.ReteBoundary.construct(ReteBoundary.java:550)
    at org.eclipse.viatra2.gtasm.patternmatcher.incremental.rete.boundary.ReteBoundary.accessProduction(ReteBoundary.java:483)
    at org.eclipse.viatra2.gtasm.patternmatcher.incremental.rete.matcher.ReteEngine.accessMatcher(ReteEngine.java:182)
    at org.eclipse.viatra2.emf.incquery.runtime.api.impl.BaseGeneratedMatcher.accessMatcher(BaseGeneratedMatcher.java:43)
    at org.eclipse.viatra2.emf.incquery.runtime.api.impl.BaseGeneratedMatcher.<init>(BaseGeneratedMatcher.java:37)
    at grabats.grabats.GrabatsMatcher.<init>(GrabatsMatcher.java:64)
    at grabats.grabats.GrabatsMatcher.<init>(GrabatsMatcher.java:52)
    at es.modelum.test.Test.testGrabatsXMIIncQuery(Test.java:938)
    at es.modelum.test.Test.main(Test.java:192)

I would be very grateful if you could please help me.

Javier

[István Ráth]

Dear Javier,

welcome to the list :-)

As for your issue, the main problem is that you are trying to run IncQuery in a standalone Java application, which is not supported (at the moment, this may change in the future). Instead, we recommend you to run this as an OSGi (RCP) application, as described at http://viatra.inf.mit.bme.hu/incquery/new/examples/application

Then, the exception will go away. This page also includes some example code which you may find useful, as it demonstrates the recommended way of using the IncQuery Java API.

(I know that this exception is not a very good indication of this problem, we'll try to improve on this shortly.)

An additional remark about your code: as far as I have noticed, you are trying to do a performance benchmark. I highly recommend you to read http://viatra.inf.mit.bme.hu/performance as it discusses a number of common issues that need to be taken into consideration. 

cheers

Istvan

--
You received this message because you are subscribed to the Google Groups "EMF-IncQuery Users" group.
To view this discussion on the web visit https://groups.google.com/d/msg/incquery-users/-/RbsfVPqaaxMJ.
To post to this group, send email to incquer...@googlegroups.com.
To unsubscribe from this group, send email to incquery-user...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/incquery-users?hl=en.

--

István

[IBWT]

Thank you for your quick response, Istvan.

I will try to run IncQuery inside an RCP application although it would be a disadvantage on performance against other solutions I am currently benchmarking.

I have already read your suggestions on performance benchmarking but I do not find them convincing. Based on my little experience on benchmarking, I have found that a Java application may experiment peaks on its memory usage that are not reflected by the totalMemory() method (e.g. intermediate data structures built for querying or other purposes that are freed before the end of the execution). I do not know if IncQuery doesn't produce such peaks, but anyways I find more reliable using JConsole to visually trace the memory usage of a Java application.

Best regards,

Javier

El martes, 10 de julio de 2012 11:44:55 UTC+2, István Ráth escribió:

Dear Javier,

welcome to the list :-)

As for your issue, the main problem is that you are trying to run IncQuery in a standalone Java application, which is not supported (at the moment, this may change in the future). Instead, we recommend you to run this as an OSGi (RCP) application, as described at http://viatra.inf.mit.bme.hu/incquery/new/examples/application

Then, the exception will go away. This page also includes some example code which you may find useful, as it demonstrates the recommended way of using the IncQuery Java API.

(I know that this exception is not a very good indication of this problem, we'll try to improve on this shortly.)

An additional remark about your code: as far as I have noticed, you are trying to do a performance benchmark. I highly recommend you to read http://viatra.inf.mit.bme.hu/performance as it discusses a number of common issues that need to be taken into consideration. 

cheers

Istvan

To post to this group, send email to incquery-users@googlegroups.com.
To unsubscribe from this group, send email to incquery-users+unsubscribe@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/incquery-users?hl=en.

--

István

[István Ráth]

 

I will try to run IncQuery inside an RCP application although it would be a disadvantage on performance against other solutions I am currently benchmarking.

In what way you expect the OSGi environment to be a performance disadvantage? If your code is written well, then once it gets to the point where measurements are taken, no OSGi/RCP-specific "extras" (e.g. lazy loading) should interfere with your results.

 

I have already read your suggestions on performance benchmarking but I do not find them convincing.

There is much more to that page than just notes regarding the memory overhead. I hope you find the rest useful.

 

Based on my little experience on benchmarking, I have found that a Java application may experiment peaks on its memory usage that are not reflected by the totalMemory() method (e.g. intermediate data structures built for querying or other purposes that are freed before the end of the execution). I do not know if IncQuery doesn't produce such peaks, but anyways I find more reliable using JConsole to visually trace the memory usage of a Java application.

In our benchmarking activities, we were always interested in the "steady state" memory impact of a tool, not the heap transients that might occur in intermittent execution phases. You are right in pointing out that such transients cannot be observed with the totalMemory() method, but they also be easily omitted on profiler charts (such as the one that JConsole uses) due to aliasing effects. Additionally, as these effects are transients, they are heavily affected by non-deterministic sources such as garbage collection anomalies (the GC algorithms rely on non-deterministic heuristics, to my best knowledge). In summary, in order to include deterministically reproducible results (i.e. ones that can be reliable measured in several experiments, and not just visually observed on a JConsole screen), we opted to include the "steady-state" results in scientific benchmarking papers.

(This is not to argue that your method is incorrect.)

cheers

Istvan

[IBWT]

I understand your approach but since the trade-off between used memory and speed is crucial for my experiments, I need to consider those transients too (the ones produced by the executed program, not the ones of the JVM or the GC), because an unnoticed transient of, for instance, 800MB may have a strong impact on the performance of an application with a memory limit of 1GB, although at the end of the execution the average used memory has been 400MB. Transient analysis helps me finding the optimal balance between memory and time.

Anyway, I suppose that both approaches are valid, it just depends on the goal you seek.

Best regards,

Javier.

[István Ráth]

Anyway, I suppose that both approaches are valid, it just depends on the goal you seek.

I agree. In fact, we are in the middle of optimization work (with special regard to memory consumption), so we would be very grateful if you could share your findings with us.

best regards

Istvan

[IBWT]

Hello Istvan.

I have downloaded the 2 projects for standalone execution but when I try to run the application one as an Eclipse Application it throws this exception:

!ENTRY org.eclipse.osgi 4 0 2012-07-11 12:05:22.661
!MESSAGE Application error
!STACK 1
java.lang.RuntimeException: No application id has been found.
    at org.eclipse.equinox.internal.app.EclipseAppContainer.startDefaultApp(EclipseAppContainer.java:242)
    at org.eclipse.equinox.internal.app.MainApplicationLauncher.run(MainApplicationLauncher.java:29)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:353)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:180)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:629)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:584)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1438)
    at org.eclipse.equinox.launcher.Main.main(Main.java:1414)

I googled it and found that sometimes is caused by using a wrong JRE, but I am running it on JRE 1.7. What might be the problem?

Thanks in advance,

Javier.

[Zoltán Ujhelyi]

Dear Javier,

there are usually two reasons for this exception:

1. for some reason the plug-in defining the selected Eclipse application extension point is not present in the run configuration
2. some dependencies for the same plug-in are not present

To check this, open the Run configurations dialog (e,g, Run/Run configurations...), look for your Eclipse application type run configuration, go to the plug-ins page, and check the following
 1. Your plug-in with the application is enabled.
 2. The org.eclipse.viatra2.patternlanguage.emf.ui project is not enabled - that would introduce some ui dependencies which could fail the execution.
 3. Pressing the Validate plug-ins button does not report any errors.

I hope, this helps,
Zoltán Ujhelyi

[IBWT]

Thank you Zoltán for your response.

Now I am getting this error:

!ENTRY org.eclipse.osgi 4 0 2012-07-11 12:25:12.864

!MESSAGE Application error
!STACK 1

java.lang.NullPointerException
    at org.eclipse.viatra2.emf.incquery.application.IncQueryHeadless.execute(IncQueryHeadless.java:74)
    at org.eclipse.viatra2.emf.incquery.application.IncQueryApplication.start(IncQueryApplication.java:73)
    at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)

    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:353)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:180)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:629)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:584)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1438)
    at org.eclipse.equinox.launcher.Main.main(Main.java:1414)

I have debugged it and the problem seems to be that this is returning null because contributedMatcherFactories is empty:

public static IMatcherFactory getMatcherFactory(String patternFqn) {
        if(contributedMatcherFactories.containsKey(patternFqn)) {
            return contributedMatcherFactories.get(patternFqn);
        }
        return null;
    }

[IBWT]

Sorry, It was a silly problem with plugin dependencies.

Thank you for your support.

[IBWT]

Hello again.

I have finally executed my test but incQuery manages itself to overload the 8GB RAM of my computer. I have followed your performance advices, but when I query the grabats pattern against my smallest model (set0.xmi, JDTAST instance with ~70k elements), incQuery claims all the memory and gets stuck.

I am attaching my test projects In case you want to run them by yourselves. I am using an i7-...@3.70GHz with 8GB of RAM running Eclipse Juno over JRE 1.7 and W7x64.

[István Ráth]

Hi,

thanks for the feedback, we'll take a look and shortly get back to you.

Istvan

[István Ráth]

Hi Javier,

try changing 

pattern methodWithReturnType(M : MethodDeclaration, S : String)= {

MethodDeclaration.returnType(M, R);

SimpleType(R);

SimpleType.name.fullyQualifiedName(R, S2);

check(S2.equals(S));

}

in your code to

pattern methodWithReturnType(M : MethodDeclaration, S : String)= {

MethodDeclaration.returnType(M, R);

SimpleType(R);

SimpleType.name.fullyQualifiedName(R, S);

//check(S2.equals(S));

}

This will avoid the "cartesian product" issue that is discussed at http://viatra.inf.mit.bme.hu/performance#Optimizing_queries_and_transformations

("Linking by edges...")

A (minor) further performance gain can be expected if you compact your minor subpatterns into the main pattern as follows:

package grabats

import "org.amma.dsl.jdt.dom"

import "org.amma.dsl.jdt.primitiveTypes"

//pattern isStatic(M : Modifier)= {

// Modifier.static(M, true);

//}

//

//pattern isPublic(M : Modifier)= {

// Modifier.public(M, true);

//}

//pattern methodOfType(T : TypeDeclaration, M : MethodDeclaration)= {

// TypeDeclaration.bodyDeclarations(T, M);

//}

pattern methodWithReturnType(M : MethodDeclaration, S : String)= {

MethodDeclaration.returnType(M, R);

SimpleType(R);

SimpleType.name.fullyQualifiedName(R, S);

//check(S2.equals(S));

}

pattern grabats(T : TypeDeclaration, M : MethodDeclaration)= {

//find methodOfType(T, M);

TypeDeclaration.bodyDeclarations(T, M);

TypeDeclaration.name.fullyQualifiedName(T, S);

find methodWithReturnType(M, S);

MethodDeclaration.modifiers(M, MF1);

MethodDeclaration.modifiers(M, MF2);

MF1 != MF2 ;

//find isPublic(MF1);

Modifier.public(M, true);

//find isStatic(MF2);

Modifier.static(M, true);

}

After these changes, the evaluation on set0.xmi (on my computer) is instantaneous.

Istvan

[IBWT]

I've applied those changes and the evaluation is instantaneous, but the query is wrong since it doesn't show any matches.

[IBWT]

Ok now it works, you just wrote Modifier.public(M, true) instead of (MF1, true) and (MF2, true). Now it takes 946s, which is fairly nice. Thank you so much for your help.

[István Ráth]

Sorry, I've made a minor mistake in the last (flattened) pattern, it should correctly read

pattern grabats(T : TypeDeclaration, M : MethodDeclaration)= {

//find methodOfType(T, M);

TypeDeclaration.bodyDeclarations(T, M);

TypeDeclaration.name.fullyQualifiedName(T, S);

find methodWithReturnType(M, S);

MethodDeclaration.modifiers(M, MF1);

MethodDeclaration.modifiers(M, MF2);

MF1 != MF2 ;

//find isPublic(MF1);

Modifier.public(MF1, true);

//find isStatic(MF2);

Modifier.static(MF2, true);

}

Then grabats.grabats has 1 match on your model (on my computer).

Istvan