Submission of encapsulated version of our instances linking tool

111 views
Skip to first unread message

Bill Happi

unread,
Sep 21, 2022, 9:25:54 AM9/21/22
to Ontology Alignment Evaluation Initiative (OAEI)
Hello everyone, 

please on which platform should I submit the encapsulated version of ours instances linking tool?

Thanks
Bill

Ernesto Jimenez-Ruiz

unread,
Sep 21, 2022, 11:08:39 AM9/21/22
to Bill Happi, Ontology Alignment Evaluation Initiative (OAEI)
Hi Bill

Are you participating in the MELT tracks or the HOBBIT tracks? Submission to Hobbit tracks should be made directly via the HOBBIT platform.

The deadline has however passed for the OAEI 2022 campaign (apart from the Bio-ML track).

Ernesto


--
You received this message because you are subscribed to the Google Groups "Ontology Alignment Evaluation Initiative (OAEI)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ontology-alignment-evaluatio...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/ontology-alignment-evaluation-initiative-oaei/1b527c87-5b68-4692-acd8-6e7dd2369429n%40googlegroups.com.


--
Ernesto Jiménez-Ruiz
Lecturer in Artificial Intelligence

Department of Computer Science
School of Science & Technology
City, University of London

Bill Happi

unread,
Sep 22, 2022, 7:15:30 AM9/22/22
to Ontology Alignment Evaluation Initiative (OAEI)
Hi Ernesto, 

Hobbit tracks

Bill

Ernesto Jimenez-Ruiz

unread,
Sep 22, 2022, 7:46:46 AM9/22/22
to Bill Happi, Ontology Alignment Evaluation Initiative (OAEI)

Bill Happi

unread,
Sep 22, 2022, 10:32:47 AM9/22/22
to Ontology Alignment Evaluation Initiative (OAEI)
I have some evaluation of these data available here : https://github.com/BillGates98/DLinker

My tool was already saved inside the campaign under the name "DLinker" 

Thanks
Bill

Ernesto Jimenez-Ruiz

unread,
Sep 22, 2022, 10:58:59 AM9/22/22
to Bill Happi, jsaveta, Ontology Alignment Evaluation Initiative (OAEI)
Hi Nina

Could you assist Bill? He is participating in the Hobbit tracks with DLinker.

Ernesto

jsaveta

unread,
Sep 22, 2022, 11:30:25 AM9/22/22
to Ernesto Jimenez-Ruiz, Bill Happi, Ontology Alignment Evaluation Initiative (OAEI)
Hi Ernesto,
I have already executed the challenges as I did not find the DLinker system integrated into the HOBBIT platform. 
If you check both OAEI SPIMBENCH benchmark and Spatial Benchmark Version 2.0 in the platform (https://master.project-hobbit.eu/benchmarks)
there is no DLinker system integrated.

If Bill can integrate them soon I can re-run the challenges (this requires some action from Micha). 
A system integration tutorial can be found here: https://hobbit-project.github.io/system_integration.html 
and LogMap integration that I personally find the most useful is here: https://gitlab.com/ernesto.jimenez.ruiz/logmap-hobbit

DLinker is supposed to participate in both challenges SPIMBENCH and Spatial? 

Best Regards,
Nina

Bill Happi

unread,
Sep 22, 2022, 4:05:23 PM9/22/22
to Ontology Alignment Evaluation Initiative (OAEI)
Hi all, 
please can the python code be encapsulated by the system adapter? if so, how? i could look into it quickly and get back to you in short time.
About the registration of my tool, it has been registered to the OAEI 2022 campaign(confert madam cassia trojahn), I currently have a version encapsulated with MELT that I was supposed to submit unfortunately I don't see on which platform to submit it nevertheless I have already done evaluations on SPATEN and SPIMBENCH HOBBIT which are available on my git that you can consult(https://github.com/BillGates98/DLinker).

I am 100% available to put me online if I am ever late

Thanks
Bill

Bill Happi

unread,
Sep 22, 2022, 4:13:28 PM9/22/22
to Ontology Alignment Evaluation Initiative (OAEI)
Hi all,

the link for tool registration is unavailable : http://tiny.cc/oaei21-participation

Thanks
Bill

hertli...@googlemail.com

unread,
Sep 23, 2022, 3:41:47 AM9/23/22
to Ontology Alignment Evaluation Initiative (OAEI)
Hi Bill,

in case you have a MELT version of your tool, you can submit it to HOBBIT with the help of the tutorial here:
an example of a python matcher for hobbit can be found here:

Best regards
Sven

Bill Happi

unread,
Sep 23, 2022, 3:53:41 AM9/23/22
to Ontology Alignment Evaluation Initiative (OAEI)
Hi sven,
 I'll start right away

Thanks 
Bill

Bill Happi

unread,
Sep 23, 2022, 7:02:09 AM9/23/22
to Ontology Alignment Evaluation Initiative (OAEI)

Bill Happi

unread,
Sep 23, 2022, 8:00:22 AM9/23/22
to Ontology Alignment Evaluation Initiative (OAEI)

I have already submitted my tool under the name "DLinker-external-python-matcher-hobbit-1.0". 

Thanks for all
Bill 

jsaveta

unread,
Sep 23, 2022, 8:36:43 AM9/23/22
to Bill Happi, Ontology Alignment Evaluation Initiative (OAEI)
Hi Bill,
I can see your system now under the benchmarks in the Hobbit platform but when I try to execute an experiment it crashes. 
You need to run some experiments (for both benchmarks) and make sure that the system adapter is implemented correctly and the experiments run properly.

* We do not need to spam the whole ontology group. You can privately answer the emails :)

Regards,
Nina

Bill Happi

unread,
Sep 23, 2022, 9:51:36 AM9/23/22
to Ontology Alignment Evaluation Initiative (OAEI)

Bill Happi

unread,
Sep 23, 2022, 10:25:35 AM9/23/22
to Ontology Alignment Evaluation Initiative (OAEI)
this line "<cmd><shell>java -cp "${project.build.finalName}.${project.packaging}:lib/*" org.hobbit.core.run.ComponentStarter de.uni_mannheim.informatik.dws.melt.receiver_hobbit.HobbitWrapper</shell></cmd>" in pom.xml, create problem because inside the "main" folder we have "DemoPythonMatcher" which can received data from any external source as URL, by different with "...HobbitWrapper" which need a system adapter to received the data. 

please, Can we take us online to solve the problem? 

Thanks
Bill

jsaveta

unread,
Sep 23, 2022, 10:25:58 AM9/23/22
to Bill Happi, Ontology Alignment Evaluation Initiative (OAEI)
The output must be the same as the training datasets that are given here:  https://hobbit-project.github.io/OAEI_2022.html

Bill Happi

unread,
Sep 24, 2022, 9:10:57 AM9/24/22
to Ontology Alignment Evaluation Initiative (OAEI)
Hi all, I have developed a java adapter but when building the image, an error pops up telling me that my account on https://git.project-hobbit.eu/users/sign_in is blocked.
please how can i unblock it?
Thanks
Bill

Bill Happi

unread,
Sep 24, 2022, 12:07:38 PM9/24/22
to Ontology Alignment Evaluation Initiative (OAEI)
Hi to all, I have submitted my image "DLinker-external-python-matcher-hobbit-1.0"  I am waiting for your returns about my the evaluations. 

Thanks
Bill

Bill Happi

unread,
Sep 24, 2022, 12:13:05 PM9/24/22
to Ontology Alignment Evaluation Initiative (OAEI)
Good evening to all, it seems to me that there is a problem with the platform. I launched the evaluation of the image and it crashed until it became untraceable and my account suddenly disappeared.

Bill

jsaveta

unread,
Sep 24, 2022, 12:22:40 PM9/24/22
to Bill Happi, Ontology Alignment Evaluation Initiative (OAEI)
Dear Bill, 

You need to make sure that your system adapter is implemented correctly and not just upload the docker image. Thus, you have to run some experiments for each benchmark, make changes in your adapter if needed until it runs. I tested your latest image for the small dataset of spimbench benchmark and "The benchmarked system could not be created.". So the system is not implemented correctly. 

Regards, 
Nina

Bill Happi

unread,
Sep 24, 2022, 12:27:42 PM9/24/22
to Ontology Alignment Evaluation Initiative (OAEI)
Thanks madame, can i send you by code  ? i need some return of you thanks

Bill

Bill Happi

unread,
Sep 24, 2022, 12:28:52 PM9/24/22
to Ontology Alignment Evaluation Initiative (OAEI)
there is way to test my image locally please ? 

jsaveta

unread,
Sep 24, 2022, 1:08:07 PM9/24/22
to Bill Happi, Ontology Alignment Evaluation Initiative (OAEI)
It is quite complicated to test it locally so I suggest to test it online. 

Bill Happi

unread,
Sep 24, 2022, 1:26:30 PM9/24/22
to Ontology Alignment Evaluation Initiative (OAEI)
it seems that my account on https://master.project-hobbit.eu/home has been deleted or blocked.

now i have updated my image and i need to push it again by i have this message "denied: requested access to the resource is denied".
what should i do?

once make i will wait for your permission to run evaluations.

Thanks
Bill

jsaveta

unread,
Sep 24, 2022, 2:10:06 PM9/24/22
to Bill Happi, Ontology Alignment Evaluation Initiative (OAEI)
Your account is not blocked I just tested it. Are you sure you are using the correct credentials?
You do not need permission to run experiments. After you successfully run some experiments and make sure everything is running smoothly I will re-run the challenge including DLinker.

Please try again to follow all the steps correctly and debug the system adapter and we can check again on Monday. 



Bill Happi

unread,
Sep 24, 2022, 6:44:01 PM9/24/22
to Ontology Alignment Evaluation Initiative (OAEI)
I see that the adapter launches well and perhaps receives the data. So I wonder if the server supports processes that launch sub-processes. Because I execute the python command with the script of my tool as input from the files received in the redefined methods.

I was able to execute a version of my image that is just waiting. I attach to this message a screenshot

Thank you for everything
Bill 
experiments.png

jsaveta

unread,
Sep 25, 2022, 2:03:26 AM9/25/22
to Bill Happi, Ontology Alignment Evaluation Initiative (OAEI)
The system is not just waiting, has crashed. Click the link with the experiment number (https://master.project-hobbit.eu/experiments/1664054304758) and download the system log in order to debug it. 

Bill Happi

unread,
Sep 25, 2022, 2:54:28 AM9/25/22
to Ontology Alignment Evaluation Initiative (OAEI)
I just made an update thank you for the new feedback. thank you for your availability especially.

Thank you
Bill

Bill Happi

unread,
Sep 25, 2022, 3:32:11 AM9/25/22
to Ontology Alignment Evaluation Initiative (OAEI)
I made a statement about 'spimbench_small_2018' as the channel delivering the source data and 'spimbench_task' as the channel delivering the target data that I found in the system logs to point my adapter to the data coming from those. I then updated my image online.
 
Thanks
Bill

jsaveta

unread,
Sep 25, 2022, 3:50:41 AM9/25/22
to Bill Happi, Ontology Alignment Evaluation Initiative (OAEI)


On Sun, Sep 25, 2022, 10:32 Bill Happi <bill...@gmail.com> wrote:
I made a statement about 'spimbench_small_2018' as the channel delivering the source data and 'spimbench_task' as the channel delivering the target data that I found in the system logs to point my adapter to the data coming from those.


Sorry but I don't get what do you mean. You just need to implement your system adapter and the benchmark will provide you with the datasets automatically. You also need to check that the system is running correctly for the Spatial benchmark. 

You do not need to inform me for each step that you are doing. When it runs correctly just let me know. 

Nina


Bill Happi

unread,
Sep 26, 2022, 4:43:32 PM9/26/22
to Ontology Alignment Evaluation Initiative (OAEI)
Good evening to all, 

Sorry for the delay, thanks for the patience. I finalized the adapter and I tested it, it runs and produces the expected output(.rdf) but I notice that the benchmark orders the return of the results as soon as the adapter is launched. You can test it again('Dlinker Tool-1.0' on the spatial benchmark and spimbench). I am waiting for your feedback.

I also join the link containing the logs of my tool(system log) :
https://master.project-hobbit.eu/experiments/1664223040836

Overall the tool is already ready to work on the benchmark, especially if the tool orders the sending of the results in a respected time frame as on the platform(60 minutes) before asking for the closing of the systems.

Thanks
Bill

Bill Happi

unread,
Sep 26, 2022, 4:53:51 PM9/26/22
to Ontology Alignment Evaluation Initiative (OAEI)
for precision you can look at line 10 and 11 of the log file of my system, the immediate closure of the adapter before sending the results obtained.

Thanks
Bill 

Bill Happi

unread,
Sep 26, 2022, 5:33:31 PM9/26/22
to Ontology Alignment Evaluation Initiative (OAEI)
test only spatial benchmark sorry

Thank
Bill

jsaveta

unread,
Sep 26, 2022, 6:15:38 PM9/26/22
to Bill Happi, Ontology Alignment Evaluation Initiative (OAEI)
Dear Bill, 
The benchmark cannot be changed as all the other participants can complete the experiments without an issue.
 
As an experiment is not returning results it means that the system is not correctly implemented and ready to participate into the challenge yet. So please make sure that you produce the output in the exact same format as given in the training datasets (https://hobbit-project.github.io/OAEI_2022.html). Spimbench requires .rdf as output and Spatial .nt fomat (check also the content of the files). 



Bill Happi

unread,
Sep 27, 2022, 5:45:55 AM9/27/22
to Ontology Alignment Evaluation Initiative (OAEI)

Hello everyone, the problem I raise here is a finding from the analysis of logs.

https://master.project-hobbit.eu/experiments/1664268258511
 
in the file '1664268258511 benchmark log.txt', line 226-227

2022-09-27T08:46:32.473Z git.project-hobbit.eu:4567/jsaveta1/spatialtaskgenerator:2.0 spatialtaskgenerato-ea858bdefe8f408c9443ada0ed433ba3.1.311tbckmkearo32somgrpuf8o [pool-1-thread-1] INFO org.hobbit.core.components.AbstractTaskGenerator - Received signal to finish.
2022-09-27T08:46:32.473Z git.project-hobbit.eu:4567/jsaveta1/spatialbenchmarkcontroller:2.0 spatialbenchmarkcontrolle-36c7d169591b4d9895e5aad7f51b41cb.1 .tivbiaumfrelvyykfvv7412yl [main] INFO org.hobbit.spatialbenchmark.platformConnection.BenchmarkController - Waiting for the task generators to finish their work.

as soon as it waits for the system to finish its work it sends a termination signal without waiting for the results.

In my logs I have the results in the requested format but they arrive after the termination signal.

I would like to know if this is related to the fact that I have a guest account?
Thanks
Bill

Bill Happi

unread,
Sep 27, 2022, 5:48:59 AM9/27/22
to Ontology Alignment Evaluation Initiative (OAEI)
concerning the content of the files, everything is okay on my side because I have the file size of my result which is calculated very well.

Thanks
Bill

jsaveta

unread,
Sep 27, 2022, 5:56:43 AM9/27/22
to Bill Happi, Ontology Alignment Evaluation Initiative (OAEI)
Can you please send the source code of your system adapter?

Bill Happi

unread,
Sep 27, 2022, 6:25:31 AM9/27/22
to Ontology Alignment Evaluation Initiative (OAEI)

Bill Happi

unread,
Sep 29, 2022, 6:50:35 AM9/29/22
to Ontology Alignment Evaluation Initiative (OAEI)
Hi everyone, I hope you are well

I would just like to reassure myself that I have not been forgotten

Kind regards
Bill

jsa...@gmail.com

unread,
Sep 29, 2022, 9:09:23 AM9/29/22
to Ontology Alignment Evaluation Initiative (OAEI)
Hi Bill,
We found two issues in the code:
1. Issue: the system adapter initializes it's super class twice. Please remove the second call to init().
2. Issue: the logging is not configured properly. Hence, there are some classes that in case of an error are not able to log the error. Please execute the following steps to fix the problem:

1. Replace the dependencies part in your pom.xml with the following lines:

    <dependencies>
        <dependency>
            <groupId>de.uni-mannheim.informatik.dws.melt</groupId>
            <artifactId>matching-base</artifactId>
            <version>${matching.version}</version>
        </dependency>
        <dependency>
            <groupId>org.hobbit</groupId>
            <artifactId>core</artifactId>
            <version>1.0.19</version> <!-- 1.0.10 1.0.9-SNAPSHOT -->
            <exclusions>
                <exclusion>
                    <groupId>xml-apis</groupId>
                    <artifactId>xml-apis</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-reload4j</artifactId>
            <version>1.7.33</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient -->
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <!--            <version>4.2.6</version>-->
            <version>4.3.6</version>
            <type>jar</type>
            <exclusions>
                <exclusion>
                    <groupId>commons-logging</groupId>
                    <artifactId>commons-logging</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
    </dependencies>
   
2. Move the log4j.properties file from src/properties into the src/main/properties directory.

After these changes, you should rebuild the project, recreate the docker image, upload it to the platform and run a test experiment.

Bill Happi

unread,
Sep 29, 2022, 10:55:55 AM9/29/22
to Ontology Alignment Evaluation Initiative (OAEI)

I did the update and pushed my image,
I found that the adapter terminates after sending the results and receives the shutdown signal very well. However, I saw other suspicious lines in the benchmark logs.

Files: https://master.project-hobbit.eu/experiments/1664457965902

in the benchmark log file,

- from line 186-203 :
it seems that the channel to receive the results does not exist so the data is sent but does not reach the destination channel.

2022-09-29T14:34:57.944Z rabbitmq:3.6-management rabbitm-2179007129904d28a1f27dce3109dfd1.1.2m03ke93s8vh4uim7blri1ie4 operation queue.declare caused a channel exception not_found: no queue 'hobbit.system-evalstore.1664457965902' in vhost '/'

Kind regards
Bill

michaasr...@gmail.com

unread,
Sep 29, 2022, 11:21:53 AM9/29/22
to Ontology Alignment Evaluation Initiative (OAEI)
Hello Bill,

my name is Michael and I am one of the maintainers of the HOBBIT platform. I am sure that we can figure out where the problem comes from.
First, I am surprised that the logging still shows a problem in your log. If you open the system log, you will see the following three lines at the bottom:
2022-09-29T13:35:03.588Z git.project-hobbit.eu:4567/billhappi/dlinker-tool:latest dlinker-tool-37a8a84f46584067a7760fbf6fba60c7.1.33ilvmdute173v6lwtd35aqh8 log4j:WARN No appenders could be found for logger (org.hobbit.core.components.AbstractCommandReceivingComponent).
2022-09-29T13:35:03.588Z git.project-hobbit.eu:4567/billhappi/dlinker-tool:latest dlinker-tool-37a8a84f46584067a7760fbf6fba60c7.1.33ilvmdute173v6lwtd35aqh8 log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
2022-09-29T13:35:03.588Z git.project-hobbit.eu:4567/billhappi/dlinker-tool:latest dlinker-tool-37a8a84f46584067a7760fbf6fba60c7.1.33ilvmdute173v6lwtd35aqh8 log4j:WARN Please initialize the log4j system properly.

These lines mean that the logging won't work and our classes can not tell us whether problems occur or not. So I would suggest to start with this issue.

Could you please commit your latest changes to github? That makes it easier for us to follow your changes and do some testing with the code.
You can also write directly to me so that we do not spam all the other people in this list.

Cheers,
Michael Röder

Bill Happi

unread,
Sep 29, 2022, 3:45:47 PM9/29/22
to Ontology Alignment Evaluation Initiative (OAEI)
hi michael,

I wrote to you by mail

Kind regards
Bill

Bill Happi

unread,
Oct 7, 2022, 6:38:36 AM10/7/22
to Ontology Alignment Evaluation Initiative (OAEI)
Hi everyone,

- Results for "Dlinker Tool-1.0 for Spatial" : spatial benchmark(OVERLAPS)
  https://master.project-hobbit.eu/experiments/1665073856825
- Results for "Dlinker Tool-1.0" for spimbench
The tools are ready.

Thank you all
Bill

jsaveta

unread,
Oct 7, 2022, 6:57:31 AM10/7/22
to Bill Happi, Ontology Alignment Evaluation Initiative (OAEI)
Hi Bill,
So I assume that you are done with the changes and we can re-run the challenges including DLinker. 
If not, you can make changes the latest until Sunday but as you can understand we will be out of office if you need something.

I would suggest running an experiment with 200 instances for Spatial (this is the size of the small track) and make sure it runs 
and that the time performance is sufficient for your system. 
Regarding SPIMBENCH you already ran the small dataset and I believe you checked the F-Measure, Recall and Precision it returned. 

We will run the challenges on Monday and make the results public most probably on Tuesday 18th. 

Regards,
Nina

Bill Happi

unread,
Oct 7, 2022, 8:56:32 AM10/7/22
to jsaveta, Ontology Alignment Evaluation Initiative (OAEI)
I will continue working on it until Sunday to ensure all will be right. 

Kind Regards
Bill
Reply all
Reply to author
Forward
0 new messages