Duplicate thumbnails in a response

0 views
Skip to first unread message

lawrence mcdonell

unread,
Mar 11, 2017, 6:58:33 AM3/11/17
to Europeana API forum
Hi,
I noticed that in a query like that below, the response contains a couple of duplicate thumbnails, as well as their titles. I wonder is there a method to check for unique thumbnails and/or titles in the query, otherwise that would mean a few round trips to get them?

James Morley

unread,
Mar 11, 2017, 9:24:30 AM3/11/17
to europe...@googlegroups.com
Hi Lawrence,

I may not be looking close enough, but can you give an example where the edmPreview is duplicated?


On 11 March 2017 at 11:58, lawrence mcdonell <lawrence...@gmail.com> wrote:
Hi,
I noticed that in a query like that below, the response contains a couple of duplicate thumbnails, as well as their titles. I wonder is there a method to check for unique thumbnails and/or titles in the query, otherwise that would mean a few round trips to get them?

--
Visit Europeana Labs for API Documentation, Open Datasets, and our Apps Showcase - http://labs.europeana.eu
---
You received this message because you are subscribed to the Google Groups "Europeana API forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to europeanaAPI+unsubscribe@googlegroups.com.
To post to this group, send email to europe...@googlegroups.com.
Visit this group at https://groups.google.com/group/europeanaAPI.
For more options, visit https://groups.google.com/d/optout.

Message has been deleted
Message has been deleted
Message has been deleted
Message has been deleted

lawrence mcdonell

unread,
Mar 11, 2017, 9:35:00 PM3/11/17
to Europeana API forum
Sorry, my bad ;{

A general question:  I am curious about the default sorting of results from search.json searches. In the query below, why are the first results all seem medieval with institution of Bodleian Library Oxford Uni.


On Sunday, March 12, 2017 at 1:24:30 AM UTC+11, James Morley wrote:
Hi Lawrence,

I may not be looking close enough, but can you give an example where the edmPreview is duplicated?
On 11 March 2017 at 11:58, lawrence mcdonell <lawrence...@gmail.com> wrote:
Hi,
I noticed that in a query like that below, the response contains a couple of duplicate thumbnails, as well as their titles. I wonder is there a method to check for unique thumbnails and/or titles in the query, otherwise that would mean a few round trips to get them?

--
Visit Europeana Labs for API Documentation, Open Datasets, and our Apps Showcase - http://labs.europeana.eu
---
You received this message because you are subscribed to the Google Groups "Europeana API forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to europeanaAPI...@googlegroups.com.

James Morley

unread,
Mar 12, 2017, 3:38:39 PM3/12/17
to europe...@googlegroups.com
Lawrence

From what I can make out, your questions are as follows:

Q: Could I ask you a related question: with a query that has media=true, is it possible to get a record without media?

A: It shouldn't be, but someone still at Europeana might like to confirm that. Of course one thing that can happen is that there is a temporary lack of access to the media file as these aren't hosted by Europeana but simply link back to teh provider's site via the edmIsShownBy that they submitted; linked with that, sometimes they change the urls but don't tell Europeana (or do redirects etc)

Q: Could I ask you a related question: what parameters should I use to only get results with edmPreview thumbnails?
media=true I guess doesn't mean that there will be an edmPreview?

A: As per http://labs.europeana.eu/api/media-search use thumbnail=true

Q: Is there a spreadsheet or other human-readable resource to access dataset names with id's and/or descriptions to make browsing of collections easier?

A: I presume in this context you're referring to the datasets on http://labs.europeana..eu/data ? There's not, but it's a nice idea. They also each used to have a link to an older version of the API Console (which was deprecated for other reasons I won't go into) and the new Swagger console at http://labs.europeana.eu/api/console, although better in so many ways, lacks the ability to link to it with preformed queries.  It would be really nice to have that resource you describe that had all the API parameters in order to replicate the datasets. I'll pass that on to my former colleagues in Europeana Labs

Q: A general question:  I am curious about the default sorting of results from search.json searches. In the query below, why are the first results all seem medieval with institution of Bodleian Library Oxford Uni..

A: I'm afraid I am not too familiar with the mystery of Solr sorting! Typically it will rank based on the frequency and positioning of any term, but that can be influenced by a number of factors, including boosts (positive or negative) in the configuration, and creation/update dates and so on. I'll flag this to someone who will hopefully be able to give you a more precise answer (I'm not sure anyone could be less precise than me!)

Best, James


For more options, visit https://groups.google.com/d/optout..

--
Visit Europeana Labs for API Documentation, Open Datasets, and our Apps Showcase - http://labs.europeana.eu
---
You received this message because you are subscribed to the Google Groups "Europeana API forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to europeanaAPI+unsubscribe@googlegroups.com.

lawrence mcdonell

unread,
Mar 13, 2017, 7:11:56 AM3/13/17
to Europeana API forum
Thanks very much for the fulsome answers James, they are very useful. 
In regard to sorting I guess I am just curious, without necessarily knowing Solr as you say, what some rules of thumb might be. The query I asked about is a simple search on athens with images in the result. Of all the millions (?) of matches there may be in the Europeana collections, we get the medieval set from Oxford. I suppose that is simple what you get when using such a basic query ...

Gordea Sergiu

unread,
Mar 13, 2017, 7:33:11 AM3/13/17
to europe...@googlegroups.com

Dear Lawrence,

 

I’m not sure what you mean by:

“. Of all the millions (?) of matches there may be in the Europeana collections, we get the medieval set from Oxford. I suppose that is simple what you get when using such a basic query ...”

 

 

1.      Please pay attention that you use some filters in your query: like thumbnail= true and IMAGE_SIZE=large.  

https://europeana.eu/api/v2/search.json?wskey=NbAjJewQk&encoding=json&profile=rich&thumbnail=true&IMAGE_SIZE=large&query=athens

 

… and the query returns "totalResults": 31674

 

2.     If you remove the thumbnail and image_size constraints (which seem to be equally restrictive on this example)

https://www.europeana.eu/api/v2/search.json?wskey=NbAjJewQk&encoding=json&profile=rich&query=athens

.. than you get "totalResults": 79078,

 

3.      If you want to get images that ae about Athens, you can use a query like this:

http://www.europeana.eu/portal/de/search?view=grid&q=what%3Aathens&f%5BTYPE%5D%5B%5D=IMAGE&per_page=96

 

You get only 650 results, but it is likely that this is closer to what you really want to search (If I understood your search purpose correctly).

 

In the results there are some items without thumbnails, but these are still linking to images that are available on content provider’s website…

Like the following record for example:

http://www.europeana.eu/portal/de/record/9200105/wellcome_historical_images_V0010650.html?q=what%3Aathens

 

https://wellcomeimages.org/indexplus/image/V0010650.html

 

 

Br,

 

Sergiu Gordea

Timothy Hill

unread,
Mar 13, 2017, 1:16:24 PM3/13/17
to Europeana API forum
Hi, Lawrence,

Regarding sorting ... while the algorithm for field-weighting is complex (we're using the BM25F ranking method, if you want to get into the gory details), the answer is essentially that the various British Library manuscript records refer to Athens prominently in their Description field; and these description fields are relatively short, meaning that the reference counts for a great deal. If you page a little further into the result set, you'll find several images of Edinburgh (the 'Athens of the North') with references to Athens in their title and description field; but these are both fairly lengthy, so the references to Athens are accorded less weight by the algorithm.

That said, we are always trying to improve our search results and our search-result ranking. So I'm curious to know what your ideal result would have been from this search term: what are you hoping to find when you enter 'Athens' as your search?

We get a lot of geographical, urban-oriented queries of this nature: a sense of what our users are looking for would be very helpful, if you can spare a moment.

Thanks,

Tim Hill
Search Engineer
Europeana

lawrence mcdonell

unread,
Mar 13, 2017, 8:23:27 PM3/13/17
to europe...@googlegroups.com
Hi Gordea,
Thank you that is very helpful, sorry about my vagueness. I am a newbie, that is why; the docs are a bit intimidating. Would you be good enough to tell me please:
The difference between using the /api/v2 and /portal?
Could you point to the docs for the parameters you are using: 



To unsubscribe from this group and stop receiving emails from it, send an email to europeanaAPI+unsubscribe@googlegroups.com.

To post to this group, send email to europe...@googlegroups.com.
Visit this group at https://groups.google.com/group/europeanaAPI.
For more options, visit https://groups.google.com/d/optout.

--
Visit Europeana Labs for API Documentation, Open Datasets, and our Apps Showcase - http://labs.europeana.eu
---
You received this message because you are subscribed to a topic in the Google Groups "Europeana API forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/europeanaAPI/UhVDt0ilzt4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to europeanaAPI+unsubscribe@googlegroups.com.

lawrence mcdonell

unread,
Mar 13, 2017, 8:26:27 PM3/13/17
to europe...@googlegroups.com
Thanks very much. I am still getting a sense of it, very ignorant. But for example, would there be a way to get the query behind the pages at the http://labs.europeana.eu/data page, but get the results not wrapped in html?

--
Visit Europeana Labs for API Documentation, Open Datasets, and our Apps Showcase - http://labs.europeana.eu
---
You received this message because you are subscribed to a topic in the Google Groups "Europeana API forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/europeanaAPI/UhVDt0ilzt4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to europeanaAPI+unsubscribe@googlegroups.com.
To post to this group, send an email to europe...@googlegroups.com.

Gordea Sergiu

unread,
Mar 14, 2017, 5:06:33 AM3/14/17
to europe...@googlegroups.com

Hi Lawrence,

 

1.      For the query syntax you can find good guidelines in the labs pages (query, search, media search):

http://labs.europeana.eu/api/query

http://labs.europeana.eu/api/search

http://labs.europeana.eu/api/media-search

 

2.      The portal performs the search through the search api, so .. everything you can make in portal can be translated into api requests.

Even if there are some small differences between the portal requests and api requests, (with regard to the parameters) it is fairly easy to transform one request in another.

Therefore, I recommend using the portal to build the proper searches for your purposes and then you can transform them in API queries

 

3.      I’m not sure which programming languages you are using, you might want to get a look into the europeana-client libraries:

http://labs.europeana.eu/apps?category=client-library&page=1

I think that some of them are outdatet on not maintained anymore, however you can try to reuse and improve some code there.

From what I know the ruby client is up to date: http://labs.europeana.eu/apps/europeana-ruby

The java version is almost up to date, but the version you can find in github   https://github.com/europeana/europeana-client which is forked from the one listed in the APP pages.

 

BR,

 

Sergiu

 

 

 

Von: europe...@googlegroups.com [mailto:europe...@googlegroups.com] Im Auftrag von lawrence mcdonell
Gesendet: Dienstag, 14. März 2017 01:24
An: europe...@googlegroups.com
Betreff: Re: {Europeana API forum} Duplicate thumbnails in a response

 

Hi Gordea,

Thank you that is very helpful, sorry about my vagueness. I am a newbie, that is why; the docs are a bit intimidating. Would you be good enough to tell me please:

The difference between using the /api/v2 and /portal?

Could you point to the docs for the parameters you are using: 

 

 

On Mon, Mar 13, 2017 at 10:33 PM, Gordea Sergiu <Sergiu...@ait.ac.at> wrote:

Dear Lawrence,

 

I’m not sure what you mean by:

“. Of all the millions (?) of matches there may be in the Europeana collections, we get the medieval set from Oxford. I suppose that is simple what you get when using such a basic query ...”

 

 

1.      Please pay attention that you use some filters in your query: like thumbnail= true and IMAGE_SIZE=large.  

https://europeana.eu/api/v2/search.json?wskey=NbAjJewQk&encoding=json&profile=rich&thumbnail=true&IMAGE_SIZE=large&query=athens

 

… and the query returns "totalResults": 31674

 

2.     If you remove the thumbnail and image_size constraints (which seem to be equally restrictive on this example)

https://www.europeana.eu/api/v2/search.json?wskey=NbAjJewQk&encoding=json&profile=rich&query=athens

.. than you get "totalResults": 79078,

 

3.      If you want to get images that ae about Athens, you can use a query like this:

http://www.europeana.eu/portal/de/search?view=grid&q=what%3Aathens&f%5BTYPE%5D%5B%5D=IMAGE&per_page=96

 

You get only 650 results, but it is likely that this is closer to what you really want to search (If I understood your search purpose correctly).

 

In the results there are some items without thumbnails, but these are still linking to images that are available on content provider’s website…

Like the following record for example:

http://www.europeana.eu/portal/de/record/9200105/wellcome_historical_images_V0010650.html?q=what%3Aathens

 

https://wellcomeimages.org/indexplus/image/V0010650.html

 

 

Br,

 

Sergiu Gordea

 

You received this message because you are subscribed to a topic in the Google Groups "Europeana API forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/europeanaAPI/UhVDt0ilzt4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to europeanaAPI...@googlegroups.com.


To post to this group, send email to europe...@googlegroups.com.
Visit this group at https://groups.google.com/group/europeanaAPI.
For more options, visit https://groups.google.com/d/optout.

lawrence mcdonell

unread,
Mar 15, 2017, 7:52:03 PM3/15/17
to Europeana API forum
Thanks very much Sergiu and  that is very helpful.

To post to this group, send email to europ...@googlegroups.com.

lawrence mcdonell

unread,
Mar 15, 2017, 7:52:40 PM3/15/17
to Europeana API forum
Thanks Timothy; no ideal result in mind yet, just groping my way.
Reply all
Reply to author
Forward
0 new messages