PhD Student: Problem compiling LIRE (downloaded from git) using IntelliJ Community 2016.2 (opencv_java249 problem)

[Nelson Silva]

Hello,

I am a PhD student trying to get started with LIRE.

I've just used Intellij to connect to github and get the current LIRE repository.

1) The project compiled without any problem, however when I try to run the TestProperties test, I always get this error:  java.lang.UnsatisfiedLinkError: no opencv_java249 in java.library.path

1.1) I already added this line to Intellij-->Run-->edit configurations-->VM Options (for TestProperties): -Djava.library.path='lib/opencv_java249' -ea

2) Also, I cannot execute any of the sample applications. For example for the liredemo app, using Intellij and after rebuilding the entire project, the LireDemoFrame.java does not show the run icon, but instead a red clock icon in this file.

Can someone please help me get started with LIRE using Intellij and/or Netbeans? 

Thank you so much in advance.

best,

Nelson

[Mathias Lux]

Hi Nelson!

As for 1.1 it might be the case that the file is not found. Check if from the current directory the path to the dll is really the one you gave it with the argument. 

As for 2) the sample applications are separated from the LIRE main project. I typically set them up as separate modules. If you add the source folders, make sure you include the libraries from the actual projects. 

regards,

Mathias

--
Sie erhalten diese Nachricht, weil Sie in Google Groups E-Mails von der Gruppe "lire-dev" abonniert haben.
Wenn Sie sich von dieser Gruppe abmelden und keine E-Mails mehr von dieser Gruppe erhalten möchten, senden Sie eine E-Mail an lire-dev+unsubscribe@googlegroups.com.
Weitere Optionen finden Sie unter https://groups.google.com/d/optout.

--

Priv.-Doz. Dr. Dipl.-Ing. Mathias Lux

Associate Professor at Klagenfurt University, Austria

http://tinyurl.com/mlux-itec ... contact and cv

LIRE image search engine - http://www.lire-project.net

[Nelson Silva]

Hello Matthias,

thank you for your fast answer.

I still could not figure out problem 1.1 in my Ubuntu system.

I see some new changes in the git repo, I will clone it again and try again.

As for problem 2, I followed your instructions and it worked.

thank you,

Nelson

--
Sie erhalten diese Nachricht, weil Sie in Google Groups ein Thema der Gruppe "lire-dev" abonniert haben.
Wenn Sie sich von diesem Thema abmelden möchten, rufen Sie https://groups.google.com/d/topic/lire-dev/uAE4i_s3WUM/unsubscribe auf.
Wenn Sie sich von dieser Gruppe und allen Themen dieser Gruppe abmelden möchten, senden Sie eine E-Mail an lire-dev+unsubscribe@googlegroups.com.

Weitere Optionen finden Sie unter https://groups.google.com/d/optout.

--

best,

Nelson Silva

[Nelson Silva]

Hello again Matthias,

I have changed to netbeans and now everything compiles just fine.

I would like to index a folder with images and calculate similarity distances between all images (I want to create a distance matrix to feed an hierarchical clustering algorithm).

Can you point me to the right directions? The Liredemo seems a good start, nevertheless if you can give me some hints I would appreciate.

Thank you.

best,

Nelson

On Wed, Nov 16, 2016 at 2:24 PM, Mathias Lux <ml...@itec.aau.at> wrote:

--

Sie erhalten diese Nachricht, weil Sie in Google Groups ein Thema der Gruppe "lire-dev" abonniert haben.
Wenn Sie sich von diesem Thema abmelden möchten, rufen Sie https://groups.google.com/d/topic/lire-dev/uAE4i_s3WUM/unsubscribe auf.

Wenn Sie sich von dieser Gruppe und allen Themen dieser Gruppe abmelden möchten, senden Sie eine E-Mail an lire-dev+unsubscribe@googlegroups.com.

Weitere Optionen finden Sie unter https://groups.google.com/d/optout.

--

best,

Nelson Silva

[Mathias Lux]

Hi!

Yesterday I've uploaded a new version of the simpleapplication employing gradle. I've tested it with IDEA and it seems to work fine. You can find it here:

https://github.com/dermotte/LIRE/releases/tag/gradle

If you choose to use the SimpleApplication package, you'll be able to find the ParalellIndexing.java example [1]. It actually fixes the first part, it gives you an easy way to index a whole directory.

The second part is trickier, but there's hope :)

You can use the searcher and iterate over all images in the index (see line 92, which uses the first document in the index for search [2]). You have to iterate over all images in the index, and then output the similarity score into some kind of matrix, I'll leave that up to you :) Make sure you  raise the number of expected search results in line 86 to the number of image in the index or above.

Another approach is to use the feature directly, like in [3]. You make a list of features and then do a double for loop computing the similarity between them. Then the matrix comes naturally :)

regards,

  mathias

[1] https://github.com/dermotte/LIRE/blob/master/samples/simpleapplication/src/main/java/net/semanticmetadata/lire/sampleapp/ParallelIndexing.java

[2] https://github.com/dermotte/LIRE/blob/master/samples/simpleapplication/src/main/java/net/semanticmetadata/lire/sampleapp/Searcher.java#L92

[3] https://github.com/dermotte/LIRE/blob/master/samples/simpleapplication/src/main/java/net/semanticmetadata/lire/sampleapp/CreateARFFFile.java

[Nelson Silva]

Hello,

Once more, thank you so much for the fast reply.
I had a deeper look at the liredemo code, and then I realized that all my questions were answered there :)

Nevertheless, I still have some doubts regarding the following requirements:

- I have a gui, that shows pictures to users, these pictures are subdivided in small regions (these should correspond to features for example of the global similarity calculation)

- now I would like to show similar images next to each other (global similarity)

These first, parts seem clear to me, with the exception of how to access the local features structure used for the similarity calculation...

- next, I want the user to select interest local regions in one or more images and search for other images with the same local features

How can I achieve this? Global Similarity, selection of interesting features, and then execute a local features images search...how?

Thank you.

Best,

Nelson

--

best,

Nelson Silva

[Mathias Lux]

Hi!

In such a case I'd go for local features. They are typically stored using a bag of visual words approach (or VLAD, but it's a similar idea)

IMHO you'd should go with the SIMPLE features for a first try. Do the extraction, aggregation, etc on th images themselves, then try to use cropped images as a query. It may show up interesting results. The SIMPLE introduction should be available here: https://github.com/dermotte/LIRE/blob/master/src/main/docs/developer-docs/docs/index.md

However, my connection is currently so slow that it won't load, otherwise I'd give you more detailed links.

regards,

mathias

[Nelson Silva]

Hello,

thank you, Matthias, for the info.

I am still not very sure how to use the code. I want to divide an image into e.g., 8X8 = 64 areas, get local features for these 64 areas.

Then I want to select 10 of these areas and search for other similar images with similar local features (to the 10 selected) in any of the 64 local areas available on those images? 

Do I have some wrong assumptions here? Maybe it is not currently possible to use the library like this?

I really thank you for your help. I am preparing some new work ideas for a new PhD paper.

best,

Nelson

[Mathias Lux]

Hi Nelson!

I'm not sure how deep your knowledge is regarding local features, so maybe here comes too much or too few details, but I try :)

local features are handled like this in LIRE

local features are extracted from a single image -> aggregator builds vocabulary using bag of visual words -> each image's local features are used to create one bag of visual words representation per image. 

Your trick would be to either fit your approach in this processing chain, or change the chain to fit your needs. 

(i) fit it in

consider using your ten areas as a single image and use this to query the classically indexed images. It should work fine enough, but it depends on your use case how good the results actually are for you

(ii) change the process to just extract and store the local features and then query the sets of local features from the ten areas (aka query set q containing multiple local features) with the local features of the images (aka set I_k for Image k, also containing multiple local features per set). You'd then handle index access and comparisons (eg.  by earth movers distance) yourself.

regards,

Mathias

[Nelson Silva]

Dear Matthias,

thank you much.

1) Regarding the "fit in" option, everything is clear to me.

2) Regarding the "change the process option", I understand the work involved. 

I am not familiar with the earth movers distance method though (I've checked it of course). I am not sure yet how it would be applied in this second option.

For my problem, I would definitively start by experimenting with the first option (fit in).

Right now I have a much more basic problem :), I can run the Liredemo with no problem, as well as the Indexer and searcher examples from the simpleapplication project. 

However, I am having problems running the IndexingAndSearchWithLocalFeatures.java example.

------

The error is: 

Exception in thread "main" java.lang.NoClassDefFoundError: org/opencv/core/Core

at net.semanticmetadata.lire.imageanalysis.features.local.opencvfeatures.CvSurfExtractor.init(CvSurfExtractor.java:92)

at net.semanticmetadata.lire.imageanalysis.features.local.opencvfeatures.CvSurfExtractor.<init>(CvSurfExtractor.java:78)

at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:423)

at java.lang.Class.newInstance(Class.java:442)

at net.semanticmetadata.lire.indexers.parallel.ExtractorItem.<init>(ExtractorItem.java:71)

at net.semanticmetadata.lire.indexers.parallel.ParallelIndexer.addExtractor(ParallelIndexer.java:527)

at net.semanticmetadata.lire.sampleapp.IndexingAndSearchWithLocalFeatures.index(IndexingAndSearchWithLocalFeatures.java:115)

at net.semanticmetadata.lire.sampleapp.IndexingAndSearchWithLocalFeatures.main(IndexingAndSearchWithLocalFeatures.java:72)

-----

What I did, was to create a new Gradle task (I am using Netbeans and Ubuntu) to run the example as follows:

----

task runIndexLocalSLocalFeat(type: JavaExec) {

    classpath = sourceSets.main.runtimeClasspath

    systemProperty "java.library.path", "lib/opencv" 

    main = 'net.semanticmetadata.lire.sampleapp.IndexingAndSearchWithLocalFeatures'

    // Define the directory where to find the images to index.

    args '/home/nsilva/Documents/images/europeana'

}

----

For some reason, it seems possible to find the opencv_2411.jar, but then it complains about the classes not being found.

Is this the wrong version of OpenCV for the latest release of LIRE? or am I missing some OpenCV files (.so...)?

I promise to not bother you so often after I have this running. 

I am understanding the samples and the LIRE code, and I am ready to start making some code changes to fit my problem.

Once more, thank you so much for your help!

best,

Nelson Silva

[Mathias Lux]

Hi!

Yes, it seems the native libs are missing or so, I'm not too good debugging this either :)

regards,

mathias

[Nelson Silva]

Dear Matthias,

After many struggles on my Ubuntu Machine, I have finally managed to compile the SimpleApplication (using OpenCV) on my Ubuntu machine :)

I've attached my workflow notes and some files that you might want to add to your git repo (to easy the life of others having the same problems as me).

What is included:

- OpenCV_2.4.11_Ubuntu.txt - step by step compilation of OpenCV for Ubuntu + configs for LIRE

- build.gradle - modified version (for the simpleapplication project) with an extra task to run the local features example and point to the OpenCV Lib

- opencv-2411_Ubuntu.piz (please change this file to zip extension) - it contains the necessary files to use opencv in Ubuntu with the LIRE examples

Thank you for the help, the fun will start now.

best,

Nelson Silva

[Mathias Lux]

Hi Nelson! 

Thanks a lot! I'll look into it in the evening and I'll add it to the documentation for sure. 

thanks again for sharing,

Mathias

[marcos...@gmail.com]

Olá Nelson,

Estou iniciando meus estudos de busca por similaridade e gostaria de saber se você pode compartilhar como fez para criar a matrix com as distâncias?

Muito obrigado.

[Nelson Silva]

Ola Marcos,

eu respondi a tua pergunta numa mensagem no forum do LIRE...penso eu.

No entanto da uma olhadela aqui tb: http://people.revoledu.com/kardi/tutorial/Clustering/

Especialmente no topico que diz: numerical example

Tambem podes dar uma vista de olhos em um dos meus papers: https://www.researchgate.net/publication/309479681_Visual_Exploration_of_Hierarchical_Data_Using_Degree-of-Interest_Controlled_by_Eye-Tracking

Espero que te ajude.

abraco,

Nelson Silva