Re: PSICQUIC: REST
26 views
Skip to first unread message
Bruno Aranda
May 7, 2013, 5:38:16 PM5/7/13
to Lukasz Salwinski, psic...@googlegroups.com
Hi!
To be honest, I don't remember very well. Obviously I guess it was very evident to me if I wrote that as documentation, but not anymore! From what I see in the implementation, the X-PSICQUIC-Supports-Compression is always True for the reference implementation... so not very useful. Like you I assume it indicates whether it supports gzip...
About the supports-formats one, I guess it is just another way to query the formats, not sure about the practical use either, unless it is to give all the metadata of the service without having to do a full request-response with body and all...
Cheers,
Bruno
On 7 May 2013 16:12, Lukasz Salwinski <luk...@mbi.ucla.edu> wrote:
Bruno,
do you remember, by any chance:
1) what is the meaning of X-PSICQUIC-Supports-Compression
header field ?
I'm afraid, 'Whether the service supports compression' (as
stated in the documentation) isn't too informative :o/ Is it
somehow supposed to duplicate the standard HTTP compression
functionality - see:
http://en.wikipedia.org/wiki/HTTP_compression
or is it to be used for something completely different ?
2) why is the functionality of
/search/formats
duplicated by a header field:
X-PSICQUIC-Supports-Formats
is every response to every rest query supposed to spit out
the list of supported formats ? why isn't the /service/formats
request not enough ???
thanks,
lukasz
--
-------------------------------------------------------------------------
Lukasz Salwinski PHONE: 310-825-1402
UCLA-DOE Institute for Genomics & Proteomics FAX: 310-206-3914
UCLA, Los Angeles EMAIL: luk...@mbi.ucla.edu
-------------------------------------------------------------------------
Lukasz Salwinski
May 8, 2013, 1:29:44 PM5/8/13
to psic...@googlegroups.com, Marine
On 05/08/2013 01:01 AM, Marine wrote:
> Hi Lukasz,
>
> Sorry for the delay in my answer.
>> Marine,
>> two questions:
> format (manually doing a gzip of tab25 results) to make the download of
> tab25 faster. The file format tab25-bin has been removed from the
> reference implementation 1.3 because the gzip compression should be
> handled by the tomcat server and not the psicquic webservice.
>
> If your tomcat configuration allows gzip compression, you could add to
> your response header field X-PSICQUIC-Supports-Compression=true. But I
> guess this header field is not useful and really specific to the server
> configuration so it should be removed from the next reference
> implementation/PSICQUIC specification.
ok, so it originally meant 'supports tab25-bin'
> psicquic-view to know which formats are available all the time without
> sending a new request to the webservice but then we should not have a
> method to get the formats... I will discuss that with Noemi but I think
> we are not using these header fields...
oh... there's probably a lot of other nice thingies that might be added
to the headers ;o) personally though I would prefer that only thingies
related to a particular query are provided (preferentially once - not
twice as 'supports tab25-bin' and 'suppots compression' ;o)
it' seems cleaner/easier to maintain/more efficient...
> The issue is that we need to be backward compatible in case some users
> are using these headers in their pipelines so it would be good if your
> psicquic webservice provides these headers as welll as the methods
> /search/formats. At some point we need to discuss in the psicquic google
> group forum about that issue and decide what we want to do with these
> duplicated methods/fheader fields. I guess for now, both HTTP header
> fields and REST methods have their advantages and disadvantages so we
> need to think.
>
> Thanks for reporting these inconsistencies with the PSICQUIC reference
> implementation!
>
> Cheers,
>
> Marine
ok, so the bottom line both features are not really useful but
got to stay because someone might be using them... :o/
I'm posting this to the psicquic list to make the issues visible there
in case someone tries to update the specification.
> Hi Lukasz,
>
> Sorry for the delay in my answer.
>> Marine,
>> two questions:
>>
>> 1) what is the meaning of X-PSICQUIC-Supports-Compression
>> header field ?
>>
>> I'm afraid, 'Whether the service supports compression' (as
>> stated in the documentation) isn't too informative :o/ Is it
>> somehow supposed to duplicate the standard HTTP compression
>> functionality - see:
>> http://en.wikipedia.org/wiki/HTTP_compression
>> or is it to be used for something completely different ?
>>
> I guess this header was added by Bruno because he added the tab25-bin
>> 1) what is the meaning of X-PSICQUIC-Supports-Compression
>> header field ?
>>
>> I'm afraid, 'Whether the service supports compression' (as
>> stated in the documentation) isn't too informative :o/ Is it
>> somehow supposed to duplicate the standard HTTP compression
>> functionality - see:
>> http://en.wikipedia.org/wiki/HTTP_compression
>> or is it to be used for something completely different ?
>>
> format (manually doing a gzip of tab25 results) to make the download of
> tab25 faster. The file format tab25-bin has been removed from the
> reference implementation 1.3 because the gzip compression should be
> handled by the tomcat server and not the psicquic webservice.
>
> If your tomcat configuration allows gzip compression, you could add to
> your response header field X-PSICQUIC-Supports-Compression=true. But I
> guess this header field is not useful and really specific to the server
> configuration so it should be removed from the next reference
> implementation/PSICQUIC specification.
ok, so it originally meant 'supports tab25-bin'
>> 2) why is the functionality of
>> /search/formats
>> duplicated by a header field:
>> X-PSICQUIC-Supports-Formats
>> is every response to every rest query supposed to spit out
>> the list of supported formats ? why isn't the /service/formats
>> request not enough ???
> True... I did not realise but this header field is nice for
>> /search/formats
>> duplicated by a header field:
>> X-PSICQUIC-Supports-Formats
>> is every response to every rest query supposed to spit out
>> the list of supported formats ? why isn't the /service/formats
>> request not enough ???
> psicquic-view to know which formats are available all the time without
> sending a new request to the webservice but then we should not have a
> method to get the formats... I will discuss that with Noemi but I think
> we are not using these header fields...
oh... there's probably a lot of other nice thingies that might be added
to the headers ;o) personally though I would prefer that only thingies
related to a particular query are provided (preferentially once - not
twice as 'supports tab25-bin' and 'suppots compression' ;o)
it' seems cleaner/easier to maintain/more efficient...
> The issue is that we need to be backward compatible in case some users
> are using these headers in their pipelines so it would be good if your
> psicquic webservice provides these headers as welll as the methods
> /search/formats. At some point we need to discuss in the psicquic google
> group forum about that issue and decide what we want to do with these
> duplicated methods/fheader fields. I guess for now, both HTTP header
> fields and REST methods have their advantages and disadvantages so we
> need to think.
>
> Thanks for reporting these inconsistencies with the PSICQUIC reference
> implementation!
>
> Cheers,
>
> Marine
ok, so the bottom line both features are not really useful but
got to stay because someone might be using them... :o/
I'm posting this to the psicquic list to make the issues visible there
in case someone tries to update the specification.
Bruno Aranda
May 8, 2013, 5:07:46 PM5/8/13
to psic...@googlegroups.com, Marine Dumousseau
Hi!
Now I remember more about the compression. Originally, you could just get a compressed version of mitab (the tab-bin format), so before creating a *bin version of the formats available, we created the option to compress every possible output. If I remember correctly, you can just passed the parameter compressed=y to the URL of the rest reference implementation.
This made huge savings on bandwidth, making the aggregators such as PSICQUIC View more efficient and faster. We are talking here about 15-20% reduction of the amount of data passed through the wire (at the expense of some CPU at the server to compress the output).
However, just checking at the code, this compression is not implemented in the SOLR version of the ws, whereas you can find it in the original one:
In the new version, the "compressed" parameter is not used at all in the methods:
Hence, the header parameter for X-PSICQUIC-Supports-Compression for the SOLR service should be false. Was there a specific reason why the compression was not added in the new version? Or it just was an oversight?
In any case, this means that probably no-one is using the functionality and a serious case to drop it altogether makes sense. In many cases, depending on the configuration of the server, the gzip/deflate is done automatically, though the non-browser clients require extra configuration to make use of this.
Cheers,
Bruno
--
--
You received this message because you are subscribed to the Google
Groups PSICQUIC group.
To post to this group, send email to psic...@googlegroups.com
To unsubscribe from this group, send email to
psicquic+unsubscribe@googlegroups.com
http://psicquic.googlecode.com
--- You received this message because you are subscribed to the Google Groups "psicquic" group.
To unsubscribe from this group and stop receiving emails from it, send an email to psicquic+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Lukasz Salwinski
May 8, 2013, 5:41:36 PM5/8/13
to psic...@googlegroups.com
On 05/08/2013 02:07 PM, Bruno Aranda wrote:
> Hi!
>
> Now I remember more about the compression. Originally, you could just get a
> compressed version of mitab (the tab-bin format), so before creating a *bin
> version of the formats available, we created the option to compress every
> possible output. If I remember correctly, you can just passed the parameter
> compressed=y to the URL of the rest reference implementation.
mhm.. it turns out the option actually does exist in the documantation.
> Hi!
>
> Now I remember more about the compression. Originally, you could just get a
> compressed version of mitab (the tab-bin format), so before creating a *bin
> version of the formats available, we created the option to compress every
> possible output. If I remember correctly, you can just passed the parameter
> compressed=y to the URL of the rest reference implementation.
but why bother if adding 'Accept-Encoding: gzip, deflate' will coax
a friendly server into compressing the response in a standard way ?
> This made huge savings on bandwidth, making the aggregators such as
> PSICQUIC View more efficient and faster. We are talking here about 15-20%
> reduction of the amount of data passed through the wire (at the expense of
> some CPU at the server to compress the output).
run the server with no compression but compress the outgoing traffic on the
proxy...
> However, just checking at the code, this compression is not implemented in
> the SOLR version of the ws, whereas you can find it in the original one:
>
> https://code.google.com/p/psicquic/source/browse/trunk/psicquic-webservice/src/main/java/org/hupo/psi/mi/psicquic/ws/IndexBasedPsicquicRestService.java
>
> In the new version, the "compressed" parameter is not used at all in the
> methods:
>
> https://code.google.com/p/psicquic/source/browse/trunk/psicquic-solr-ws/src/main/java/org/hupo/psi/mi/psicquic/ws/SolrBasedPsicquicRestService.java
>
> SOLR service should be false. Was there a specific reason why the
> compression was not added in the new version? Or it just was an oversight?
I guess it's because some of the web servers/proxies and servlet containers (eg
> compression was not added in the new version? Or it just was an oversight?
apache httpd ,tomcat) can be configured to compress content on demand ?
> In any case, this means that probably no-one is using the functionality and
> a serious case to drop it altogether makes sense. In many cases, depending
> on the configuration of the server, the gzip/deflate is done automatically,
> though the non-browser clients require extra configuration to make use of
> this.
one extra line:
http://stackoverflow.com/questions/7546783/uncompress-gzip-http-response-using-jersey-client-api-java
I'd guess, as the operation is pretty standard, some library/module should
be available for most of the setups...
lukasz
> Cheers,
>
> Bruno
mar...@ebi.ac.uk
May 9, 2013, 3:04:57 AM5/9/13
to Bruno Aranda, psic...@googlegroups.com, Marine Dumousseau
Hello Bruno,
We removed the tab25-bin format in the new reference implementation
because it is a better practice to use Content-Encoding: gzip which is
supported by the HTTP protocol itself.
Most if not all HTTP clients support gzip compression without any explicit
user code, so using this standard header improves transfer speed for all
clients including browser-initiated downloads.
The PSICQUIC releases that removed support for tab25-bin added the gzip
filter to Jetty. Groups using Tomcat should configure it to support gzip,
which is really something all servers should do anyway for text-based
data.
Performance is indeed improved using compression, and we preferred using
HTTP standards over non-standards protocols in the new implementation.
Cheers,
Marine
> Hi!
>
> Now I remember more about the compression. Originally, you could just get
> a
> compressed version of mitab (the tab-bin format), so before creating a
> *bin
> version of the formats available, we created the option to compress every
> possible output. If I remember correctly, you can just passed the
> parameter
> compressed=y to the URL of the rest reference implementation.
>
> This made huge savings on bandwidth, making the aggregators such as
> PSICQUIC View more efficient and faster. We are talking here about 15-20%
> reduction of the amount of data passed through the wire (at the expense of
> some CPU at the server to compress the output).
>
> However, just checking at the code, this compression is not implemented in
> the SOLR version of the ws, whereas you can find it in the original one:
>
> https://code.google.com/p/psicquic/source/browse/trunk/psicquic-webservice/src/main/java/org/hupo/psi/mi/psicquic/ws/IndexBasedPsicquicRestService.java
>
> In the new version, the "compressed" parameter is not used at all in the
> methods:
>
> https://code.google.com/p/psicquic/source/browse/trunk/psicquic-solr-ws/src/main/java/org/hupo/psi/mi/psicquic/ws/SolrBasedPsicquicRestService.java
>
> Hence, the header parameter for X-PSICQUIC-Supports-**Compression for the
>> ------------------------------**------------------------------**
>> -------------
>> -------------
>> .
>> For more options, visit
>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
>> .
>>
>>
>>
>
We removed the tab25-bin format in the new reference implementation
because it is a better practice to use Content-Encoding: gzip which is
supported by the HTTP protocol itself.
Most if not all HTTP clients support gzip compression without any explicit
user code, so using this standard header improves transfer speed for all
clients including browser-initiated downloads.
The PSICQUIC releases that removed support for tab25-bin added the gzip
filter to Jetty. Groups using Tomcat should configure it to support gzip,
which is really something all servers should do anyway for text-based
data.
Performance is indeed improved using compression, and we preferred using
HTTP standards over non-standards protocols in the new implementation.
Cheers,
Marine
> Hi!
>
> Now I remember more about the compression. Originally, you could just get
> a
> compressed version of mitab (the tab-bin format), so before creating a
> *bin
> version of the formats available, we created the option to compress every
> possible output. If I remember correctly, you can just passed the
> parameter
> compressed=y to the URL of the rest reference implementation.
>
> This made huge savings on bandwidth, making the aggregators such as
> PSICQUIC View more efficient and faster. We are talking here about 15-20%
> reduction of the amount of data passed through the wire (at the expense of
> some CPU at the server to compress the output).
>
> However, just checking at the code, this compression is not implemented in
> the SOLR version of the ws, whereas you can find it in the original one:
>
> https://code.google.com/p/psicquic/source/browse/trunk/psicquic-webservice/src/main/java/org/hupo/psi/mi/psicquic/ws/IndexBasedPsicquicRestService.java
>
> In the new version, the "compressed" parameter is not used at all in the
> methods:
>
> https://code.google.com/p/psicquic/source/browse/trunk/psicquic-solr-ws/src/main/java/org/hupo/psi/mi/psicquic/ws/SolrBasedPsicquicRestService.java
>
> SOLR service should be false. Was there a specific reason why the
> compression was not added in the new version? Or it just was an oversight?
>
> In any case, this means that probably no-one is using the functionality
> and
> a serious case to drop it altogether makes sense. In many cases, depending
> on the configuration of the server, the gzip/deflate is done
> automatically,
> though the non-browser clients require extra configuration to make use of
> this.
>
> Cheers,
>
> Bruno
> On 8 May 2013 18:29, "Lukasz Salwinski" <luk...@mbi.ucla.edu> wrote:
>
>> On 05/08/2013 01:01 AM, Marine wrote:
>>
>>> Hi Lukasz,
>>>
>>> Sorry for the delay in my answer.
>>>
>>>> Marine,
>>>> two questions:
>>>>
>>>> 1) what is the meaning of X-PSICQUIC-Supports-**Compression
> compression was not added in the new version? Or it just was an oversight?
>
> In any case, this means that probably no-one is using the functionality
> and
> a serious case to drop it altogether makes sense. In many cases, depending
> on the configuration of the server, the gzip/deflate is done
> automatically,
> though the non-browser clients require extra configuration to make use of
> this.
>
> Cheers,
>
> Bruno
> On 8 May 2013 18:29, "Lukasz Salwinski" <luk...@mbi.ucla.edu> wrote:
>
>> On 05/08/2013 01:01 AM, Marine wrote:
>>
>>> Hi Lukasz,
>>>
>>> Sorry for the delay in my answer.
>>>
>>>> Marine,
>>>> two questions:
>>>>
>>>> header field ?
>>>>
>>>> I'm afraid, 'Whether the service supports compression' (as
>>>> stated in the documentation) isn't too informative :o/ Is it
>>>> somehow supposed to duplicate the standard HTTP compression
>>>> functionality - see:
>>>> http://en.wikipedia.org/wiki/**HTTP_compression<http://en.wikipedia.org/wiki/HTTP_compression>
>>>>
>>>> I'm afraid, 'Whether the service supports compression' (as
>>>> stated in the documentation) isn't too informative :o/ Is it
>>>> somehow supposed to duplicate the standard HTTP compression
>>>> functionality - see:
>>>> or is it to be used for something completely different ?
>>>>
>>>> I guess this header was added by Bruno because he added the tab25-bin
>>> format (manually doing a gzip of tab25 results) to make the download of
>>> tab25 faster. The file format tab25-bin has been removed from the
>>> reference implementation 1.3 because the gzip compression should be
>>> handled by the tomcat server and not the psicquic webservice.
>>>
>>> If your tomcat configuration allows gzip compression, you could add to
>>> your response header field X-PSICQUIC-Supports-**Compression=true. But
>>>>
>>>> I guess this header was added by Bruno because he added the tab25-bin
>>> format (manually doing a gzip of tab25 results) to make the download of
>>> tab25 faster. The file format tab25-bin has been removed from the
>>> reference implementation 1.3 because the gzip compression should be
>>> handled by the tomcat server and not the psicquic webservice.
>>>
>>> If your tomcat configuration allows gzip compression, you could add to
>> -------------
>> Lukasz Salwinski PHONE: 310-825-1402
>> UCLA-DOE Institute for Genomics & Proteomics FAX: 310-206-3914
>> UCLA, Los Angeles EMAIL: luk...@mbi.ucla.edu
>> ------------------------------**------------------------------**
>> UCLA-DOE Institute for Genomics & Proteomics FAX: 310-206-3914
>> UCLA, Los Angeles EMAIL: luk...@mbi.ucla.edu
>> -------------
>>
>> --
>> --
>> You received this message because you are subscribed to the Google
>> Groups PSICQUIC group.
>> To post to this group, send email to psic...@googlegroups.com
>> To unsubscribe from this group, send email to
>> psicquic+unsubscribe@**googlegroups.com<psicquic%2Bunsu...@googlegroups.com>
>> --
>> --
>> You received this message because you are subscribed to the Google
>> Groups PSICQUIC group.
>> To post to this group, send email to psic...@googlegroups.com
>> To unsubscribe from this group, send email to
>>
>> http://psicquic.googlecode.com
>> --- You received this message because you are subscribed to the Google
>> Groups "psicquic" group.
>> To unsubscribe from this group and stop receiving emails from it, send
>> an
>> email to
>> psicquic+unsubscribe@**googlegroups.com<psicquic%2Bunsu...@googlegroups.com>
>> http://psicquic.googlecode.com
>> --- You received this message because you are subscribed to the Google
>> Groups "psicquic" group.
>> To unsubscribe from this group and stop receiving emails from it, send
>> an
>> email to
>> .
>> For more options, visit
>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
>> .
>>
>>
>>
>
Marine
May 9, 2013, 6:50:17 AM5/9/13
to Bruno Aranda, psic...@googlegroups.com
Hi Bruno,
I think you only answered to me so I copied your e-mail to psicquic google groups!
I agree that we should remove the unnecessary header from psicquic-solr-ws and updated the documentation. But what do you mean by unused argument in the methods? I think I removed it from the documentation 1.3 and from the psicquic-solr-ws implementation so it does not show as a supported format.
Cheers,
Marine
I think you only answered to me so I copied your e-mail to psicquic google groups!
I agree that we should remove the unnecessary header from psicquic-solr-ws and updated the documentation. But what do you mean by unused argument in the methods? I think I removed it from the documentation 1.3 and from the psicquic-solr-ws implementation so it does not show as a supported format.
Cheers,
Marine
Ok, so then we can just remove both the unnecessary header and the unused argument in the methods, and the related documentation line. Is everybody happy with this? Could do it myself later...
Cheers
Bruno
Bruno Aranda
May 9, 2013, 8:01:59 AM5/9/13
to Marine, psic...@googlegroups.com
Ah, ok, thanks for re-sending my email! What I mean looking at
I see that the String argument "compressed" in the line 139 seems not to be used? Just looking at the code from the browser though, so I may be missing something.
Cheers,
Bruno
Marine
May 9, 2013, 9:35:16 AM5/9/13
to Bruno Aranda, psic...@googlegroups.com
Hi Bruno,
Yes, you are totally right, this argument should be removed!
Feel free to update the source code and remove the header field from the documentation. I wanted to repackage and release the webservice next week with some bug fixes when indexing in SOLR (could cause duplicated interactions) and I think it may be good to integrate your changes as well with the header so I will wait for you to commit the changes before doing anything.
Thanks!
Marine
Yes, you are totally right, this argument should be removed!
Feel free to update the source code and remove the header field from the documentation. I wanted to repackage and release the webservice next week with some bug fixes when indexing in SOLR (could cause duplicated interactions) and I think it may be good to integrate your changes as well with the header so I will wait for you to commit the changes before doing anything.
Thanks!
Marine
Reply all
Reply to author
Forward
0 new messages