Re: mostSimilar Example with simple model not working

[Shilad Sen]

Thanks, Amgad.

Sorry this doesn't work. It definitely sounds like a bug!

I suspect that some of the necessary feature indices aren't getting built. Could you tell me what files exist in "dat/sr/ESA/simple"?

-Shilad

On Mon, Feb 23, 2015 at 2:13 PM, <amgad....@gmail.com> wrote:

Hi,

Im having trouble running your "mostSimilar" example using the simple model generated via the GUI. This is the error i get.

WARNING: directory /home/amadkour/dat/sr/inlinknotrain/simple does not exist; cannot read files
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.BaseSRMetric configureBase
INFO: finished base configuration of metric inlinknotrain
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/ESA/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/ESA/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.BaseSRMetric configureBase
INFO: finished base configuration of metric ESA
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.vector.LinkGenerator createBlackListSet
INFO: Skipping blacklist creation; no blacklist file specified.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/inlink/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/inlink/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.BaseSRMetric configureBase
INFO: finished base configuration of metric inlink
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.vector.LinkGenerator createBlackListSet
INFO: Skipping blacklist creation; no blacklist file specified.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/outlink/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/outlink/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.BaseSRMetric configureBase
INFO: finished base configuration of metric outlink
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/category/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/category/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.BaseSRMetric configureBase
INFO: finished base configuration of metric category
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/inlink/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/inlink/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/outlink/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/outlink/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.BaseSRMetric configureBase
INFO: finished base configuration of metric milnewitten
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/inlink/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/inlink/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/outlink/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/outlink/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.BaseSRMetric configureBase
INFO: finished base configuration of metric milnewitten
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/ensemble/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.utils.SrNormalizers isValidNormalizer
WARNING: Failed to load normalizer at /home/amadkour/dat/sr/ensemble/simple/mostSimilarNormalizer. Setting it to be invalid.
Feb 23, 2015 2:57:21 PM org.wikibrain.sr.BaseSRMetric configureBase
INFO: finished base configuration of metric ensemble
Exception in thread "main" java.lang.NullPointerException
        at org.wikibrain.sr.vector.CosineSimilarity.mostSimilarWithInvertedIndex(CosineSimilarity.java:176)
        at org.wikibrain.sr.vector.CosineSimilarity.mostSimilar(CosineSimilarity.java:137)
        at org.wikibrain.sr.vector.VectorBasedSRMetric.mostSimilar(VectorBasedSRMetric.java:176)
        at org.wikibrain.sr.ensemble.EnsembleMetric.mostSimilar(EnsembleMetric.java:86)
        at org.wikibrain.sr.BaseSRMetric.mostSimilar(BaseSRMetric.java:264)
        at org.wikibrain.sr.BaseSRMetric.mostSimilar(BaseSRMetric.java:278)
        at edu.purdue.jtests.WikiBrainMostSimilar.main(WikiBrainMostSimilar.java:38)



To reproduce the error:
-----------------------

- Download the latest JAR
- Run the GUI to download and generate the model and tick all items (except spatial) with "simple" model selected
- Create a new java project and include the JAR as its dependency, also use -c customized.conf as an input parameter
- Run the strings pairwise similarity example mentioned on the website [Works correctly]
- Run the mostSimilar example on the website [Error]

Code: http://shilad.github.io/wikibrain/tutorial/sr.html

Notes:
------
* Please note that /home/amadkour/dat/sr/ensemble/simple/mostSimilarNormalizer exists but the file size is 755 bytes. My guess the model was not created (though your GUI should have done so as i had a message indicating all was done successfully in the CSV file and the GUI? )

* My only guess i need to recreate the model manually through the commandline, yet i need to guess what the correct parameters needed are as every combination i try generates errors.


Thanks

-- Amgad

--
You received this message because you are subscribed to the Google Groups "wikibrain" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wikibrain+...@googlegroups.com.
To post to this group, send email to wiki...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/wikibrain/de3db7cc-dba0-42d5-a373-48aea60f7883%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Shilad W. Sen
Associate Professor
Mathematics, Statistics, and Computer Science Dept.
Macalester College
ss...@macalester.edu

http://www.shilad.com

https://www.linkedin.com/in/shilad

651-696-6273

[amgad....@gmail.com]

Hi Shilad,

Thanks for getting back to me on this issue. I managed to find a work-around by running:

java -cp java -cp wikibrain-withdeps-0.5.2.jar org.wikibrain.sr.SRBuilder

which built all the models. It is not clear for me if the GUI should have built the models or not, since it was able to answer the pairwise similarity query. The documentation was not very clear regarding that aspect though it did mention that models need to be built in general. If the GUI is not meant to build all models then this is not a bug. In case it should have built them then these are the two files that were only there back then:

mostSimilarNormalizer
similarityNormalizer

I think this was the case for all the other measures as well.

Hope this helps.

-- Amgad

[Shilad Sen]

Amgad,

You're right. The GUI is intended to support similarity() queries (not mostSimilar). To support mostSimilar() queries, much more computational work needs to be performed. We should update the documentation to reflect this.

-Shilad

To view this discussion on the web visit https://groups.google.com/d/msgid/wikibrain/35b05f51-0f75-4d9c-953e-30a15a2203c4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.