How often is Wikipedia re-indexed?

[Daniel Kornev]

First of all, I'm super glad to be a user of Dandelion platform. It's super cool and lovely.

However, as a new user of WikiSearch (as opposed to Entity Extraction which is integrated into our currently stealth product since last year), I've been surprised that some of the entities that are clearly in Wikipedia are hard to impossible to find using a WikiSearch API.

For instance, there is an entity I know exists in Wikipedia (was "Alice (virtual assistant)" until recently), but I couldn't find it using the WikiSearch API.

[Daniel Kornev]

Being unsure about using brackets, I've renamed article title to "Yandex Alice", but it didn't help, either:

https://en.wikipedia.org/wiki/Yandex_Alice

[kar...@taskhuman.com]

As per the link here: https://dandelion.eu/docs/ I see that April 2017 is being used currently. But this is a great question. Dandelion team, we would really appreciate if the Wikipedia data are reindexed more often to keep all the services current. I notice that the Wikipedia data are uploaded to Mediawiki servers twice a month (https://dumps.wikimedia.org/enwiki/). It would be a great value prop if team Dandelion can consider updating their indexes and models more often. 

[Steffen John]

I'd like to know this as well. For example there is an article about Quizartinib, but it is not known as an entity. 

[Aaron Phillips]

I'd also like to Dandelion re-indexing Wikipedia more frequently.

Is this something the community could do?

[Giacomo Berardi]

Hi,

your concern is rightful.

We have discussed this subject in the last months, our goal would be to define a new infrastructure for maintaining data structures, but also custom user data which is based on specific data version.

We are planning a model for satisfying all users, however most of them are heavily dependent on the Wikipedia versions in use.

We still not have a clear solution, but this topic is absolutely a priority in Dandelion development.

Cheers

Giacomo Berardi

Dandelion team

[Steffen John]

Since DBpedia isn't updating their NLP files or has decided not to share them anymore, is it save to assume that dandelion will not be updated as well? I think you still use the last available dbpedia NLP/ontology version from late 2016. Nearly all drug names since 2017 are not recognized, etc.. Is there any news you can share on this? 

[Giacomo Berardi]

Hi Steffen,

we will keep the support of DBpedia data as long as it will be compatible with Wikipedia data. However we will focus on other resources, i.e. Wikidata. 

We are still defining the future of our knowledge base structure, the goal will be to keep it constantly updated, giving priority to fresh data.

[Steffen John]

Can you share any updates on this? For example Covid19  is still not detected and many new drugs aren't either. I get the feeling Dandelion.eu is sunsetting. Can you confirm this?

[Daniel Kornev]

I echo this question.

 In the meantime, if you want to try out Entity Linking connected to the latest Wikidata KG, you can try out our Entity Linking in KBQA model available at

 http://docs.deeppavlov.ai/en/master/features/models/kbqa.html

 You can either download an entire Wikidata (and update it locally), or you can use our Entity Linking inside our KBQA model against live Wikidata APIs.

 Best Regards,

Daniel

[Giacomo Berardi]

Hi guys,  even if Dandelion seems static we are still working and taking care of it. Dandelion has been quite stable and still satisfies most of our user requirements.

We understand , however, the need of an updated knowledge base, and it is where we are focusing now. 

Maybe not at the speed of light, but we haven't abandoned Dandelion at any time.

Recent changes will help us to scale better in the development and focus on the data quality, giving the same robust API services.

Cheers

Giacomo Berardi 

Dandelion team

[Steffen John]

Well, this question has been opened in January 2019. And now you answer with a link of "new features", which basically says you remove "Custom spots", the only way, we could circumvent the totally outdated database. I am very disappointed. We actually spent quite a bit of time implementing the API in our product and have been paying monthly the last 2 years.

[Giacomo Berardi]

We are not adding new features, but radically changing our product, keeping its core and enhancing it with improvements, principally related to the infrastructure. The entity linking is working fine, it just needs a robust data flow system. We are dropping some of the features of Dandelion principally because no one is using them, so we can reallocate the effort for maintaining them on more critical aspects of Dandelion. Thank you for your patience!

Giacomo Berardi

Dandelion team

[Steffen John]

So, 9 months later. Any updates/news?

[Giacomo Berardi]

Hi Steffen,

yes , we have actively worked on Dandelion in the last months, even it is not visible on the website.

We prefer to not make announcements now, but we have definitely invested on Dandelion, so changes will come for sure in the next months.

Giacomo Berardi

Dandelion team