Continuous Evaluation of Relational Learning in Biomedicine

[Robert Hoehndorf]

Dear colleagues,

The Bio-Ontologies COSI starts to run a continuous evaluation of link
prediction methods this year which I believe many of you will find
interesting; we also plan to have a special session as part of the
Bio-Ontologies COSI @ ISMB 2021 to discuss the initial results and
future directions. Please consider participating, and share this
invitation with anybody who might be interested.

CERLIB:

Predicting relations between biological entities using machine learning
is a common and important task in computational biology. CERLIB is a new
challenge that aims to continuously evaluate state-of-the-art relation
prediction methods as new biological knowledge becomes available.

We invite you to participate in the Continuous Evaluation of:

- Link prediction on biological knowledge graphs
(http://biochallenge.bio2vec.net/challenges/view/1)

- Gene-disease prediction
(http://biochallenge.bio2vec.net/challenges/view/2)

- Drug-target associations
(KG-COVID-19<http://biochallenge.bio2vec.net/challenges/view/3)

Challenge Website: biochallenge.bio2vec.net

(First) Meeting: Special Session @ Bio-Ontologies COSI at ISMB 2021
(https://www.iscb.org/ismbeccb2021)

Overview: The analysis of biological networks has long been a central
component of computational biology. Networks and knowledge graphs form a
crucial component of life science infrastructure where hundreds of data-
and knowledge-bases have been developed. Networks and knowledge graphs
are not only used to store and retrieve information but are also used
for network- and knowledge-based analyses. One type of analysis includes
determining whether a relation holds true or false within a knowledge
base; this question can be solved deductively, inductively, or
transductively. In the past 5 years, we have witnessed a proliferation
of methods based on machine learning that address the problem of
predicting relations from graph-based knowledge. While these methods are
developed to solve tasks in many knowledge graphs, they are rarely
evaluated and compared on a variety of biological knowledge. Moreover,
every evaluation and comparison usually represents only a snapshot in
time and may depend on a specific context that includes parameters,
training/testing splits, random seeds, or various pre- and
post-processing steps, thereby making it difficult to reproduce and
compare results. Furthermore, in many cases, only “positive” predictions
are evaluated (i.e. new relations), however, some relations also turn
out to be incorrect and thus are removed from a dataset; methods that
determine whether relations are wrongly asserted are vitally important
and currently underrepresented. Biological knowledge evolves rapidly,
and the corresponding data- and knowledge-bases are updated regularly to
reflect new discoveries. This provides an opportunity for a time-based,
prospective, and continuous evaluation of relation prediction methods.

We will use biological databases that continuously update their data and
make it accessible through public SPARQL endpoints for evaluation. We
will run the same query monthly and evaluate the submissions made up to
this point.

Join the CERLIB challenge, which aims to collect and evaluate relation
predictions in life sciences using an unbiased, empirical approach based
on the growing body of biological knowledge. The challenge submission
begins February 2021. For more information and to register, visit the
challenge site: https://biochallenge.bio2vec.net

With best wishes,
Rob.