Compressing 'Request' Content from the Browser.
5 views
Skip to first unread message
vvpre...@gmail.com
Jul 11, 2008, 5:38:39 AM7/11/08
to Gears Users
Hello -
Is there (will there be?) a Gears API that allows compression of
"request" content in the browser before sending (POST) the request to
the server. I see a great use for this now that Gears enables us to
use browser side db (sqlite). We do a lot of data sync-ups of content
from the Gears db back to the server and enabling compression of the
payload potentially helps increase performance of the POSTs.
Note that I am requesting an inverse scenario of the already existing
content-encoding: gzip, mod_deflate (similar) features.
Is there any other alternatives that exists today ?
Regards,
Preetham V V
Is there (will there be?) a Gears API that allows compression of
"request" content in the browser before sending (POST) the request to
the server. I see a great use for this now that Gears enables us to
use browser side db (sqlite). We do a lot of data sync-ups of content
from the Gears db back to the server and enabling compression of the
payload potentially helps increase performance of the POSTs.
Note that I am requesting an inverse scenario of the already existing
content-encoding: gzip, mod_deflate (similar) features.
Is there any other alternatives that exists today ?
Regards,
Preetham V V
Ben Lisbakken
Jul 11, 2008, 12:22:44 PM7/11/08
to gears...@googlegroups.com
Hey Preetham --
We actually have a section on our open source hosting project dedicated to brainstorming new APIs. Here is the link, if you haven't taken a look (http://code.google.com/p/gears/w/list).
In there you can take a look at the FileSystemAPI, which might have some things you would be interested in (http://code.google.com/p/gears/wiki/FileSystemAPI). Now, just because something's on that list, it doesn't mean we are doing it for sure, but it usually means it is something that we think is a good idea, and with enough time and resources would be something we would like to implement.
-Ben
We actually have a section on our open source hosting project dedicated to brainstorming new APIs. Here is the link, if you haven't taken a look (http://code.google.com/p/gears/w/list).
In there you can take a look at the FileSystemAPI, which might have some things you would be interested in (http://code.google.com/p/gears/wiki/FileSystemAPI). Now, just because something's on that list, it doesn't mean we are doing it for sure, but it usually means it is something that we think is a good idea, and with enough time and resources would be something we would like to implement.
-Ben
Chris Prince
Jul 11, 2008, 1:30:44 PM7/11/08
to gears...@googlegroups.com
That's an interesting idea.
Have you run any tests to see how the upload-time performance differs
with different compression options?
On Fri, Jul 11, 2008 at 2:38 AM, vvpre...@gmail.com
<vvpre...@gmail.com> wrote:
>
vvpre...@gmail.com
Jul 12, 2008, 8:43:41 AM7/12/08
to Gears Users
Hi Ben -
Thanks for the link. Meanwhile, I was thinking of a JS based
compression algorithm that can help compress the payload within the
browser environment. In my current product, we have a policy where we
do not allow File access (except sqlite db of course) for our App for
security reasons.
I guess I will have to initiate a small prototype to try Huffman
compression written in JS and check the time it may take to compress a
2MB JSON payload. As long as the compression is taking reasonable
amount of time and memory (as this shall be a backend activity that
can happen on a worker and does not hold the UI thread) we must be ok
(As the sync by themselves are frequency based)...
On the Server side we do not mind receiving the compressed payload as
is and then inflate the same in a filter (Servlet). We are not
expecting Apache or Tomcat to understand the compression type and do
anything about it automatically (as it may happen today in the inverse
scenario)...
Thanks,
Preetham V V
On Jul 11, 9:22 pm, Ben Lisbakken <lisba...@google.com> wrote:
> Hey Preetham --
>
> We actually have a section on our open source hosting project dedicated to
> brainstorming new APIs. Here is the link, if you haven't taken a look (http://code.google.com/p/gears/w/list).
>
> In there you can take a look at the FileSystemAPI, which might have some
> things you would be interested in (http://code.google.com/p/gears/wiki/FileSystemAPI). Now, just because
> something's on that list, it doesn't mean we are doing it for sure, but it
> usually means it is something that we think is a good idea, and with enough
> time and resources would be something we would like to implement.
>
> -Ben
>
> On Fri, Jul 11, 2008 at 2:38 AM, vvpreet...@gmail.com <vvpreet...@gmail.com>
Thanks for the link. Meanwhile, I was thinking of a JS based
compression algorithm that can help compress the payload within the
browser environment. In my current product, we have a policy where we
do not allow File access (except sqlite db of course) for our App for
security reasons.
I guess I will have to initiate a small prototype to try Huffman
compression written in JS and check the time it may take to compress a
2MB JSON payload. As long as the compression is taking reasonable
amount of time and memory (as this shall be a backend activity that
can happen on a worker and does not hold the UI thread) we must be ok
(As the sync by themselves are frequency based)...
On the Server side we do not mind receiving the compressed payload as
is and then inflate the same in a filter (Servlet). We are not
expecting Apache or Tomcat to understand the compression type and do
anything about it automatically (as it may happen today in the inverse
scenario)...
Thanks,
Preetham V V
On Jul 11, 9:22 pm, Ben Lisbakken <lisba...@google.com> wrote:
> Hey Preetham --
>
> We actually have a section on our open source hosting project dedicated to
> brainstorming new APIs. Here is the link, if you haven't taken a look (http://code.google.com/p/gears/w/list).
>
> In there you can take a look at the FileSystemAPI, which might have some
> things you would be interested in (http://code.google.com/p/gears/wiki/FileSystemAPI). Now, just because
> something's on that list, it doesn't mean we are doing it for sure, but it
> usually means it is something that we think is a good idea, and with enough
> time and resources would be something we would like to implement.
>
> -Ben
>
vvpre...@gmail.com
Jul 12, 2008, 9:05:07 AM7/12/08
to Gears Users
We have tried manually compressing a 100MB JSON payload which gives
almost a 80% compression ratio (GZIP). This is good amount of
bandwidth saved for our enterprise app (currently takes in excess of
35mins to transfer this as is)... typically some of our syncs ranges
between of 100-300MB.
The 100MB payload gets broken down currently into smaller chunks
(based on network stats and capacity planning) and then posted to the
server. If we can achieve good compressions, we are fairly certain
that the 80% compression can be analogous to a max of 40%-50% on the
time saved (excluding the time it may take to compress).
We have to find a good JS based compression algorithm and then try
these theories out...
Regards,
Preetham V V
On Jul 11, 10:30 pm, "Chris Prince" <cpri...@google.com> wrote:
> That's an interesting idea.
>
> Have you run any tests to see how the upload-time performance differs
> with different compression options?
>
> On Fri, Jul 11, 2008 at 2:38 AM, vvpreet...@gmail.com
almost a 80% compression ratio (GZIP). This is good amount of
bandwidth saved for our enterprise app (currently takes in excess of
35mins to transfer this as is)... typically some of our syncs ranges
between of 100-300MB.
The 100MB payload gets broken down currently into smaller chunks
(based on network stats and capacity planning) and then posted to the
server. If we can achieve good compressions, we are fairly certain
that the 80% compression can be analogous to a max of 40%-50% on the
time saved (excluding the time it may take to compress).
We have to find a good JS based compression algorithm and then try
these theories out...
Regards,
Preetham V V
On Jul 11, 10:30 pm, "Chris Prince" <cpri...@google.com> wrote:
> That's an interesting idea.
>
> Have you run any tests to see how the upload-time performance differs
> with different compression options?
>
Brad Neuberg
Jul 14, 2008, 3:17:11 PM7/14/08
to gears...@googlegroups.com
Hi vvpreetham, tell us more about how you use Gears; sounds interesting.
Michael Nordman
Jul 14, 2008, 3:20:51 PM7/14/08
to gears...@googlegroups.com
This is a great feature request.
ghr.setHeader('Content-Encoding', 'gzip'); ----> compressed body when POST'ing
ghr.setHeader('Content-Encoding', 'gzip'); ----> compressed body when POST'ing
vvpre...@gmail.com
Jul 15, 2008, 2:04:40 AM7/15/08
to Gears Users
Hi Brad -
I shall partly give a very high level overview cause most of what we
are doing is kind of sensitive to IP.
We are developing a SaaS based Retail POS which can run in a
disconnected environment (courtesy Gears). The POS application is self
sufficient to conduct offline product checks, execute multitudes of
tenders (customizable), compute tax structures at the line-item and
bill-level (state and city based) and complex discount structures
(currently based on a 3 dimensional structure). The back end
Infrastructure is cloud-based to support in excess of 30,000 POS from
a single host (cloud). There are nearly 3 million articles that is
available in the article master out of which nearly 500K products with
varying depth of category, subcategory, brand, manufacturer, supplier
etc... must be assigned to a store catalog (which may be running on a
average 15 POS). Now we have customized the first time master sync of
importing 500K articles from the cloud to every single POS (SQLite).
Subsequently based on a frequency a master-delta gets copied to POS.
Also all the transactions made on each POS which is now locally stored
needs to get sync-ed to the cloud... Gears is used here a la SQLite,
Workers, Localserver (manifest based updates), JSNI (almost every inch
of Gears) etc... GWT is used for UI.
Compression and Security on the client side is utmost importance as of
now...
Regards,
Preetham V V
On Jul 15, 12:17 am, Brad Neuberg <bradneub...@google.com> wrote:
> Hi vvpreetham, tell us more about how you use Gears; sounds interesting.
>
> On Fri, Jul 11, 2008 at 2:38 AM, vvpreet...@gmail.com <vvpreet...@gmail.com>
I shall partly give a very high level overview cause most of what we
are doing is kind of sensitive to IP.
We are developing a SaaS based Retail POS which can run in a
disconnected environment (courtesy Gears). The POS application is self
sufficient to conduct offline product checks, execute multitudes of
tenders (customizable), compute tax structures at the line-item and
bill-level (state and city based) and complex discount structures
(currently based on a 3 dimensional structure). The back end
Infrastructure is cloud-based to support in excess of 30,000 POS from
a single host (cloud). There are nearly 3 million articles that is
available in the article master out of which nearly 500K products with
varying depth of category, subcategory, brand, manufacturer, supplier
etc... must be assigned to a store catalog (which may be running on a
average 15 POS). Now we have customized the first time master sync of
importing 500K articles from the cloud to every single POS (SQLite).
Subsequently based on a frequency a master-delta gets copied to POS.
Also all the transactions made on each POS which is now locally stored
needs to get sync-ed to the cloud... Gears is used here a la SQLite,
Workers, Localserver (manifest based updates), JSNI (almost every inch
of Gears) etc... GWT is used for UI.
Compression and Security on the client side is utmost importance as of
now...
Regards,
Preetham V V
On Jul 15, 12:17 am, Brad Neuberg <bradneub...@google.com> wrote:
> Hi vvpreetham, tell us more about how you use Gears; sounds interesting.
>
> wrote:
>
>
>
>
>
> > Hello -
>
> > Is there (will there be?) a Gears API that allows compression of
> > "request" content in the browser before sending (POST) the request to
> > the server. I see a great use for this now that Gears enables us to
> > use browser side db (sqlite). We do a lot of data sync-ups of content
> > from the Gears db back to the server and enabling compression of the
> > payload potentially helps increase performance of the POSTs.
>
> > Note that I am requesting an inverse scenario of the already existing
> > content-encoding: gzip, mod_deflate (similar) features.
>
> > Is there any other alternatives that exists today ?
>
> > Regards,
> > Preetham V V
>
> --
> Best,
> Brad
>
> bradneub...@google.com>
>
>
>
>
> > Hello -
>
> > Is there (will there be?) a Gears API that allows compression of
> > "request" content in the browser before sending (POST) the request to
> > the server. I see a great use for this now that Gears enables us to
> > use browser side db (sqlite). We do a lot of data sync-ups of content
> > from the Gears db back to the server and enabling compression of the
> > payload potentially helps increase performance of the POSTs.
>
> > Note that I am requesting an inverse scenario of the already existing
> > content-encoding: gzip, mod_deflate (similar) features.
>
> > Is there any other alternatives that exists today ?
>
> > Regards,
> > Preetham V V
>
> --
> Best,
> Brad
>
Brad Neuberg
Jul 15, 2008, 11:38:43 AM7/15/08
to gears...@googlegroups.com, Gears Users
How do you interface with devices on the POS side from the browser and
Gears, such as a barcode scanner?
Gears, such as a barcode scanner?
Sent from my cell phone
On Jul 14, 2008, at 11:04 PM, "vvpre...@gmail.com" <vvpre...@gmail.com
Chris Prince
Jul 15, 2008, 9:48:55 PM7/15/08
to gears...@googlegroups.com
Filed as a feature request, for tracking purposes:
http://code.google.com/p/gears/issues/detail?id=587
http://code.google.com/p/gears/issues/detail?id=587
vvpre...@gmail.com
Jul 18, 2008, 10:04:57 AM7/18/08
to Gears Users
We work on the UPOS standard drivers (JPOS and JSPOS) supported by
Nixdorf etc... Most of these drivers are OS specific and pretty much
work on any application (including browser) once configured. Gears
would not come into picture hear. Any TextBox (GWT) can accept the
input feed. [Hardware encryption is a different story though] The
trick is to make the devices which are not UPOS specific to work on
the browser (such as pole display, weighing machines, RFID etc...)...
We land up writing JS factory plugins specifically for a given -
device, browser, OS - combo... Luckily we are sticking to Ubuntu and
Firefox for all our POS platforms for now. So our configmap is still
manageable for the foreseeable future...
On Jul 15, 8:38 pm, Brad Neuberg <bradneub...@google.com> wrote:
> How do you interface with devices on the POS side from the browser and
> Gears, such as a barcode scanner?
>
> Sent from my cell phone
>
> On Jul 14, 2008, at 11:04 PM, "vvpreet...@gmail.com" <vvpreet...@gmail.com
Nixdorf etc... Most of these drivers are OS specific and pretty much
work on any application (including browser) once configured. Gears
would not come into picture hear. Any TextBox (GWT) can accept the
input feed. [Hardware encryption is a different story though] The
trick is to make the devices which are not UPOS specific to work on
the browser (such as pole display, weighing machines, RFID etc...)...
We land up writing JS factory plugins specifically for a given -
device, browser, OS - combo... Luckily we are sticking to Ubuntu and
Firefox for all our POS platforms for now. So our configmap is still
manageable for the foreseeable future...
On Jul 15, 8:38 pm, Brad Neuberg <bradneub...@google.com> wrote:
> How do you interface with devices on the POS side from the browser and
> Gears, such as a barcode scanner?
>
> Sent from my cell phone
>
Reply all
Reply to author
Forward
0 new messages