WeeWX WebP image plot test

[Paul R Anderson]

 What is WebP ?

An image format developed by Google in 2010. Quote from Google “WebP is a modern image format that provides superior lossless and lossy compression for images on the web. Using WebP, webmasters and web developers can create smaller, richer images that make the web faster.” WebP adoption has been slow but as of now in 2022 it appears that around 95% of user devices such as desktop PC’s and phones support the file format. One article here has some data on supporting browsers https://caniuse.com/webp

Given the adoption rate I thought it was time to see if WeeWX could produce plots in WebP format. It was very easy to get WeeWX to produce WebP format plots. Results were an average file size reduction of 50% percent, with little if any discernible image quality differences.

Been using only WebP plot images on my very low traffic website for a few days and haven’t heard any complaints. Would love if you could take some to try to view the site and let me know of any browser, device issues. Also what you think of image quality, to me they look like normal WeeWX plots.

Site:

https://pauland.net

Thank You,

Paul

[Tom Hogland]

Looks fine for me - Ubuntu x64, tested with Chrome, Firefox, Brave and Opera.

[vince]

I get no images shown in Safari 15.3 on an Intel MacBook Air.

[Paul R Anderson]

Hi Vince,

Thanks for trying. So I see from the can I use site that Safari is a problem for sure.

 15.3 - Partial support in Safari  limited to macOS 11 Big Sur and later

So there's that. :)

Thanks,

Paul

On Sat, Mar 5, 2022 at 11:57 AM vince <vince...@gmail.com> wrote:

I get no images shown in Safari 15.3 on an Intel MacBook Air.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/1a681fa7-a449-4311-aa8e-ee38c51d4134n%40googlegroups.com.

[vince]

Looks ok if I use chrome on the same iPad.

[Paul R Anderson]

Strange state of support for WebP that's for sure. Wish it was 100% supported but it's not. Was just on a quest , for no good reason, to reduce the download size of my main index.html file. It's been a fun learning experience. Reduced page download size from around 450kB to  70kb , and request from 45 to 15. In today's world of fast unlimited internet connections it serves little purpose I guess, other than reducing server load and page load time by a small amount.

Thanks

Paul      

On Mon, Mar 7, 2022 at 1:14 PM vince <vince...@gmail.com> wrote:

Looks ok if I use chrome on the same iPad.

--

You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/dce97591-4519-4736-a30c-66182a8ac433n%40googlegroups.com.

[Tom Hogland]

It does serve a purpose. My Vantage Pro is in my yard and weewx is in my basement, but my webpage to display it is hosted in a datacenter somewhere far, far away, so I have to sftp it there. A full batch of data uploading typically takes almost a minute to go, besides the overhead, connection time, etc. and the time weewx takes to do all it's processing. Since that happens every 5 minutes, it's not impossible for my 5-minute workload to overlap the next 5 minute interval.  If I can reduce it, then everything works better.

Tom

http://www.alaskatech.org/weewx/index.html

[Tom Keffer]

About 10 years ago, I switched to rsync and was astonished at how much faster it is. Maybe an order of magnitude! It completely solved my upload problem.

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAMwKJErNwkUektGgYrw%3DVQPMbGqTUKxwHrcBx%3DJZVhvH4yb7iw%40mail.gmail.com.

[Paul R Anderson]

Totally agree with Tom rsync is incredible if your hosted server supports it try it, you'll never go back.

Spoiled here with fiber 300 Mbps upload and download here but here are WeeWX rsync transfers for my site here 10 North of Boston Massachusetts, to my host in Newark, NJ

Mar  8 10:30:25 webserver weewx[18755] INFO weeutil.rsyncupload: rsync'd 34 files (259,081 bytes) in 0.60 seconds
Mar  8 10:35:22 webserver weewx[18755] INFO weeutil.rsyncupload: rsync'd 39 files (1,431,654 bytes) in 0.65 seconds
Mar  8 10:40:22 webserver weewx[18755] INFO weeutil.rsyncupload: rsync'd 34 files (259,557 bytes) in 0.65 seconds
Mar  8 10:45:23 webserver weewx[18755] INFO weeutil.rsyncupload: rsync'd 36 files (278,303 bytes) in 0.58 seconds
Mar  8 10:50:22 webserver weewx[18755] INFO weeutil.rsyncupload: rsync'd 39 files (1,431,670 bytes) in 0.66 seconds
Mar  8 10:55:22 webserver weewx[18755] INFO weeutil.rsyncupload: rsync'd 34 files (259,994 bytes) in 0.62 seconds
Mar  8 11:00:24 webserver weewx[18755] INFO weeutil.rsyncupload: rsync'd 45 files (296,208 bytes) in 0.63 seconds
Mar  8 11:05:22 webserver weewx[18755] INFO weeutil.rsyncupload: rsync'd 39 files (1,431,828 bytes) in 0.66 seconds
Mar  8 11:10:22 webserver weewx[18755] INFO weeutil.rsyncupload: rsync'd 34 files (259,759 bytes) in 0.61 seconds
Mar  8 11:15:22 webserver weewx[18755] INFO weeutil.rsyncupload: rsync'd 34 files (260,131 bytes) in 0.66 seconds
Mar  8 11:20:22 webserver weewx[18755] INFO weeutil.rsyncupload: rsync'd 39 files (1,431,817 bytes) in 0.64 seconds
Mar  8 11:25:22 webserver weewx[18755] INFO weeutil.rsyncupload: rsync'd 34 files (260,418 bytes) in 0.61 seconds
Mar  8 11:30:22 webserver weewx[18755] INFO weeutil.rsyncupload: rsync'd 34 files (260,343 bytes) in 0.61 seconds
Mar  8 11:35:22 webserver weewx[18755] INFO weeutil.rsyncupload: rsync'd 39 files (1,432,512 bytes) in 0.64 seconds

To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEC8BLBrxqieuE9F5MC2ZW5-CL-v3Ap5vvv%2BwhSyn6PJKA%40mail.gmail.com.

[Tom Hogland]

I thought of that, but the only supported method I can use is sftp.

On Tue, Mar 8, 2022 at 7:44 AM Paul R Anderson <pa...@pauland.net> wrote:

Totally agree with Tom rsync is incredible if your hosted server supports it try it, you'll never go back.

Spoiled here with fiber 300 Mbps upload and download here but here are WeeWX rsync transfers for my site here 10 North of Boston Massachusetts, to my host in Newark, NJ

[snip]

[vince]

You might consider finding another service provider that 'does' support rsync.

If you're comfortable setting up a small webserver, Amazon Lightsail is under $4 per month.

[paul.ba...@gmail.com]

1. SFTP can be used with compression (-C option) but I don't know if it is used or not in Weewx.

2. I tried to compress all my weewx images with tar and/or gzip.  At best I get less than 30 % reduction.  May be it is just enough for you.

3. gzip (or better the library behind it) is in fact used by ssh, sftp, tar etc. So in your case you should just be sure to use compressionwhatever the transfer protocol.

When lines were very slow, the default was always WITH compression. Now, with much, much faster lines available, the default is NO compression, as compression would slow down the process...

Regards,     PaulB

[Tom Hogland]

I don't know much about Python :-) but I see cnopts=pysftp.CnOpts() in the sftp.py script. This leads down the typical rabbit-hole of Google, grep, etc. and I find the pysftp project cookbook and added "cnopts.compression=True" in there. We'll see if it works - the cookbook says that compression is off by default. 

So far (one cycle) it looks about the same - no errors, and "26 files, 34 seconds" versus the last uploads of 21 files, 32 seconds. (Second cycle - 21 files, 30.9 seconds. Faster, but tiny...)

I may look into alternate providers, but I host this with Gandi (in Luxembourg) for $5/mo for a pre-configured and running host. As long as sftp works I'm happy to use it.