Is calculated value of console/Image rewrite failures changed

[Eric Longuemare]

Hello,

Since update to last mod_pagespeed beta, "console/Image rewrite failures" report half (a little more than 50% - fallen slowly) rewrite failure rates (after 72 hours).

has handled values changed since  https://github.com/pagespeed/mod_pagespeed/issues/1214 ?

greatings,

Eric

PS : as I'm testing some memcached conf (minimizing memory usage on recycled server) it's maybe related.

/usr/local/memcached/bin/memcached -m 768 -p 11211 -u myuser -l 127.0.0.1 -o slab_reassign,slab_automove,lru_crawler,lru_maintainer,maxconns_fast,hash_algorithm=murmur3

[Jeff Kaufman]

All #1214 was was that the javascript for the console was broken so it
wouldn't start; that shouldn't have affected statistics.

Are you saying you're seeing higher numbers for an image failure
statistic than in a previous version of PageSpeed? Which statistic?
Which earlier version?

> --
> You received this message because you are subscribed to the Google Groups
> "mod-pagespeed-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to mod-pagespeed-di...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/mod-pagespeed-discuss/b7cba27b-5325-43af-b4f3-6ecfe899e5cf%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Eric Longuemare]

Hello Jeff,

All #1214 was was that the javascript for the console was broken so it
wouldn't start; that shouldn't have affected statistics.

Are you saying you're seeing higher numbers for an image failure
statistic than in a previous version of PageSpeed?

Yes, from 1.9.xx and from before 1.10.33 (that broke console)

 

 Which statistic?

Console/ image failure

Which earlier version?

I have followed all alphas then last beta released.
So I can't see console/image failure since december release :  1.10.33.0-7562

I use this memcached conf :  /usr/local/memcached/bin/

memcached -m 768 -p 11211 -u myuser -l 127.0.0.1 -o slab_reassign,slab_automove,lru_crawler,lru_maintainer,maxconns_fast,hash_algorithm=murmur3

since 1rst week of november 2015.
I can;t remember to have seen such a high rate of image failure in console (except on 1rst cache warm day)

mod_pagespeed config :

Version: 14: on

Filters
cw	Collapse Whitespace
jc	Combine Javascript
gp	Convert Gif to Png
jp	Convert Jpeg to Progressive
jw	Convert Jpeg To Webp
pj	Convert Png to Jpeg
ws	When converting images to WebP, prefer lossless conversions
dj	Defer Javascript
hw	Flushes html
io	In-place optimize for browser
idp	Insert DNS Prefetch
js	Jpeg Subsampling
ll	Lazyload Images
pr	Prioritize Critical Css
rj	Recompress Jpeg
rp	Recompress Png
rw	Recompress Webp
rc	Remove Comments
cf	Rewrite Css
jm	Rewrite External Javascript
jj	Rewrite Inline Javascript
cp	Strip Image Color Profiles
md	Strip Image Meta Data

Options
  AvoidRenamingIntrospectiveJavascript (aris)  False
  CssInlineMaxBytes (ci)                       10240
  EnableCachePurge (euci)                      True
  EnableRewriting (e)                          1
  FileCacheCleanIntervalMs (afcci)             3600000
  FileCachePath (afcp)                         /var/cache/mod_pagespeed/
  FileCacheSizeKb (afc)                        4096000
  ImplicitCacheTtlMs (ict)                     31098734000
  LoadFromFileCacheTtlMs (lfct)                31098734000
  LogDir (ald)                                 /var/log/pagespeed
  LRUCacheByteLimit (alcb)                     16384
  LRUCacheKbPerProcess (alcp)                  512
  MemcachedServers (ams)                       localhost:11211
  RequestOptionOverride (roo)                  mykey
  RewriteDeadlinePerFlushMs (rdm)              50
  RewriteLevel (l)                             Optimize For Bandwidth
  SslCertDirectory (assld)                     /etc/ssl/certs
  StatisticsLogging (asle)                     True
  SupportNoScriptEnabled (snse)                False
  UrlSigningKey (usk)                         mykey

Domain Lawyer
  http://www.domain.tld/ Auth

[Steve Hill]

If you're suspicious of your memcache configuration, the logical thing is to switch it back and see if that solves the errors. But unless your memcache is totally broken, I don't think that would affect your image failure statistic. If there anything from mod_pagespeed in your apache logs? Could you send a complete dump of all the statistics?

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/292c5aca-9ba6-470a-8c4d-1a5178b27da1%40googlegroups.com.

[Eric Longuemare]

Hello,

>If there anything from mod_pagespeed in your apache logs

No, only some low file read as usual (very few - 50 since five days / for about 200 000 image hits ) , no other error, nor in apache error.log and pagespeed message_hystory..

I check the other things tomorrow.

Thanks,

Eric

[Eric Longuemare]

Some values from console/json response :

{
    "timestamps": [1459961518873, 1459962123206, 1459962730182, 1459963333741],
    "variables": {
        "cache_backend_hits": [391536, 392548, 393426, 394613],
        "cache_backend_misses": [50113, 50184, 50230, 50343],
        "cache_expirations": [61216, 61335, 61445, 61552],
        "css_filter_blocks_rewritten": [172, 172, 172, 172],
        "css_filter_parse_failures": [0, 0, 0, 0],
        "image_norewrites_high_resolution": [124, 124, 124, 124],
        "image_rewrites": [32079, 32128, 32148, 32160],
        "image_rewrites_dropped_decode_failure": [0, 0, 0, 0],
        "image_rewrites_dropped_due_to_load": [35611, 35611, 35611, 35611],
        "image_rewrites_dropped_mime_type_unknown": [14, 14, 14, 14],
        "image_rewrites_dropped_nosaving_noresize": [436, 436, 436, 436],
        "image_rewrites_dropped_nosaving_resize": [0, 0, 0, 0],
        "image_rewrites_dropped_server_write_fail": [0, 0, 0, 0],
        "javascript_blocks_minified": [277817, 278243, 278677, 279132],
        "javascript_minification_failures": [0, 0, 0, 0],
        "num_cache_control_not_rewritable_resources": [7, 7, 7, 7],
        "num_cache_control_rewritable_resources": [111038, 111187, 111252, 111352],
        "resource_url_domain_acceptances": [2841992, 2846659, 2851640, 2856613],
        "resource_url_domain_rejections": [75080, 75117, 75182, 75250],
        "serf_fetch_failure_count": [0, 0, 0, 0],
        "serf_fetch_request_count": [97, 97, 97, 97]
    }
}

[Eric Longuemare]

The "big value" is "image_rewrites_dropped_due_to_load": [35611, 35611, 35611, 35611],

This value is stable then drop from a few thousand.

I can't find "Too busy" messages in mod_pagespeed message history when the value of "image_rewrites_dropped_due_to_load" drops.

I check apache access logs.

Thanks,

Eric

[Eric Longuemare]

Are all "mage_rewrites_dropped_due_to_load" gives "too busy" logs in message_history ?

[Steve Hill]

Right, the "Too busy to rewrite image" message should appear whenever image_rewrites_dropped_due_to_load goes up. That definitely could be caused by too small of a cache; If the cache entry expires, your server will have to re-write the image again.

Could you try changing your memcache settings back and see if it goes away?

On Wed, Apr 6, 2016 at 1:40 PM, Eric Longuemare <opal...@gmail.com> wrote:

Are all "mage_rewrites_dropped_due_to_load" gives "too busy" logs in message_history ?

--

You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/9d548dbf-73e1-494c-bcef-fd1fab4c8b9d%40googlegroups.com.

[Eric Longuemare]

Thanks,

I'm monitoring to try to catch one "big drop" in "images_rewrites_dropped_due_to_load" (none for this night), and will change memcached config as this was done.

Eric

memcached stats :

STAT pid 1580
STAT uptime 523024
STAT time 1460015664
STAT version 1.4.25
STAT libevent 2.0.16-stable
STAT pointer_size 32
STAT rusage_user 379.083691
STAT rusage_system 108.090755
STAT curr_connections 9
STAT total_connections 598
STAT rejected_connections 0
STAT connection_structures 12
STAT reserved_fds 20
STAT cmd_get 643597
STAT cmd_set 441609
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 329934
STAT get_misses 313663
STAT delete_misses 0
STAT delete_hits 0
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 2131891977
STAT bytes_written 4433558262
STAT limit_maxbytes 805306368
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT time_in_listen_disabled_us 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 262144
STAT hash_is_expanding 0
STAT slab_reassign_rescues 0
STAT slab_reassign_evictions_nomem 0
STAT slab_reassign_inline_reclaim 0
STAT slab_reassign_busy_items 0
STAT slab_reassign_running 0
STAT slabs_moved 0
STAT lru_crawler_running 0
STAT lru_crawler_starts 6687
STAT lru_maintainer_juggles 3736942
STAT malloc_fails 0
STAT bytes 582495898
STAT curr_items 64215
STAT total_items 441609
STAT slab_global_page_pool 0
STAT expired_unfetched 0
STAT evicted_unfetched 86470
STAT evictions 89607
STAT reclaimed 0
STAT crawler_reclaimed 0
STAT crawler_items_checked 8710745
STAT lrutail_reflocked 4
STAT moves_to_cold 179835
STAT moves_to_warm 8543
STAT moves_within_lru 14894
STAT direct_reclaims 89607
END

[Eric Longuemare]

Hello,

It has reproduced about one hour ago (big drop) = about 2 thousand of "too busy" between two stats :
Checking apache access logs, I found one ip that requested about a thousand pictures in 2 minutes.

Image rewrites - 34419, 34425, 34451, 34481, 35310, 35336
Image load drops- 35662, 35662, 35662, 35662, 37842, 37842

So it's no a stat problem but mine.

Thanks,

Eric

[Steve Hill]

Glad to hear it. Good luck!

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/4223e781-c03d-4c6b-b16c-a8f332fdad69%40googlegroups.com.

[Eric Longuemare]

Some mod_pagespeed doc on "image_rewrites_dropped_due_to_load" :

https://developers.google.com/speed/pagespeed/module/console#graphs

Thanks,

Eric


[Eric Longuemare]

Hello,

https://developers.google.com/speed/pagespeed/module/reference-image-optimize#ImageMaxRewritesAtOnce

Is setting ImageMaxRewritesAtOnce (ModPagespeedImageMaxRewritesAtOnce) to 0 removes the bound as writed in pagespeed.conf file or "Do not set this option to 0 to disable image optimization – that just adds computational overhead during page load" as it is writed in online doc ?

Thanks,

Eric

[Huibao Lin]

Setting ImageMaxRewritesAtOnce to 0 means "do not rewrite any image". You can increase this parameter to allow more images to concurrently rewrite. The default is 8.

--

You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/e65ecec1-ad7d-4b2d-b9b5-ae6964007408%40googlegroups.com.

[Jeff Kaufman]

Eric is right, though: we give conflicting guidance in our default config:

# Bound the number of images that can be rewritten at any one time; this
# avoids overloading the CPU. Set this to 0 to remove the bound.
#
# ModPagespeedImageMaxRewritesAtOnce 8

(See pagespeed.conf.template)

The docs currently say "don't set it to 0" while the default conf
currently says "it's fine to set it to zero if you want to remove the
limit". Huibao: how do you want to resolve this?

> https://groups.google.com/d/msgid/mod-pagespeed-discuss/CALg8Q9ygNrXU7p4sX059cYWkDfh7sUKvxzvG%2Bi%3DKEqEhdwnpjg%40mail.gmail.com.

[Jeff Kaufman]

Filed a bug: https://github.com/pagespeed/mod_pagespeed/issues/1289

[Eric Longuemare]

Hello,

Testing ImageMaxRewritesAtOnce with different values changes nothing at all.

I'm using now last beta released : 1.11.33.0

With IPRO on.

I've tested on two apache servers : one with limited ressouces (old recycled hardware) and on another one with 8Gb memcached and 4 cores Xeon 3Ghz, SAS harddrive ....

At most,, on one requested HTML page with more than 10 pictures (html embedded thumbnails), only 9 pictures are rewrited at once. Others throws "Too busy for rewrite ...".

I have set ImageMaxRewritesAtOnce to 0, -1 and these limit is allways there, can't get rid of it.

When ImageMaxRewritesAtOnce is not set, max image rewrites/html page are 8.

So I think, if load limit is not tested by Pagespeed, there's a limit that is hardcoded somewhere.

Another point : report in console/image failure is not accurate (I think) :

example : one page with 20 pictures and 8 pictures rewrited at once.

first rewrite -> 20 ->8 pictures rewrited -> 12 pictures rewrited failure reported (Too busy)
second rewrite -> 12 -> 8 pictures rewrited -> 4 pictures rewrited failure reported (Too busy)
third rewrite -> 4 -> 4 pictures rewrited -> 0 failure reported. (Too busy)

The 16 failures are staying in image failures count,
Succes are 20.

report is 16/20 failures and stay there, (so I have 80% of failure reported but all image have been rewrited.

If I always have less than 8 pictures (thumbnails) in page, success will be 100% and console/image failure 0%

I think this is why I don't remember to have seen so high values of failure in the past.

Thanks,

Eric

Pagespeed conf :

Filters
cw	Collapse Whitespace
jc	Combine Javascript
gp	Convert Gif to Png
jp	Convert Jpeg to Progressive
jw	Convert Jpeg To Webp
pj	Convert Png to Jpeg
ws	When converting images to WebP, prefer lossless conversions
dj	Defer Javascript
hw	Flushes html
io	In-place optimize for browser
idp	Insert DNS Prefetch
js	Jpeg Subsampling
ll	Lazyload Images
pr	Prioritize Critical Css
rj	Recompress Jpeg
rp	Recompress Png
rw	Recompress Webp
rc	Remove Comments
cf	Rewrite Css
jm	Rewrite External Javascript
jj	Rewrite Inline Javascript
cp	Strip Image Color Profiles
md	Strip Image Meta Data

Options
  AvoidRenamingIntrospectiveJavascript (aris)  False
  CssInlineMaxBytes (ci)                       10240
  EnableCachePurge (euci)                      True
  EnableRewriting (e)                          1
  FileCacheCleanIntervalMs (afcci)             3600000
  FileCachePath (afcp)                         /var/cache/mod_pagespeed/
  FileCacheSizeKb (afc)                        4096000
  ImageMaxRewritesAtOnce (im)                  0
  ImplicitCacheTtlMs (ict)                     31098734000
  LazyloadImagesAfterOnload (llio)             False
  LazyloadImagesBlankUrl (llbu)                http://www.gstatic.com/psa/static/1.gif
  LoadFromFileCacheTtlMs (lfct)                31098734000
  LogDir (ald)                                 /var/log/pagespeed
  LRUCacheByteLimit (alcb)                     16384
  LRUCacheKbPerProcess (alcp)                  1024
  MemcachedServers (ams)                       localhost:11211,localhost:11212
  RequestOptionOverride (roo)                  mykey
  RewriteDeadlinePerFlushMs (rdm)              50
  RewriteLevel (l)                             Optimize For Bandwidth
  SslCertDirectory (assld)                     /etc/ssl/certs
  StatisticsLogging (asle)                     True
  SupportNoScriptEnabled (snse)                False
  UrlSigningKey (usk)                          mykey

Domain Lawyer
  http://www.tld.net/ Auth

memcached conf (at now, but a lot of conf have been tested) :
48400 394176 www-data  2538     1    9 00:00:00 /usr/local/bin/memcached -m 4096 -p 11212 -u www-data -l 127.0.0.1 -o slab_reassign,slab_automove,lru_crawler,lru_maintainer,maxconns_fast,hash_algorithm=murmur3
47248 393140 www-data  2535     1    9 00:00:00 /usr/local/bin/memcached -m 4096 -p 11211 -u www-data -l 127.0.0.1 -o slab_reassign,slab_automove,lru_crawler,lru_maintainer,maxconns_fast,hash_algorithm=murmur3

[Otto van der Schaaf]

I could reproduce that changing ImageMaxRewritesAtOnce has no effect in ngx_pagespeed as well.

In the debugger I observed that:

- the default options from the driver factory do not reflect the configured value, and remains at the default (8)

-- the value the global options from the server context does reflect the configured value, which is why the correct configured value turns up in the admin interface as that uses the global options from the server context as its source.

Created https://github.com/pagespeed/mod_pagespeed/issues/1305 for tracking this

[Eric Longuemare]

Hello @Otto,

I'm testing with your commit since this morning :

- ImageMaxRewritesAtOnce set to 0 or -1 gives back unlimited image rewrites
- setting  ImageMaxRewritesAtOnce to a value > 0 gives limited image rewrites to the number defined
- not setting ImageMaxRewritesAtOnce gives default value, 8 images rewrites at once.

Thanks,

Eric

[Otto van der Schaaf]

Actually it was Maks his commit, not mine.

Thanks for confirming that MaxImageRewritesAtOnce is now respected with that commit!

Otto

--
You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-di...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/a0c5ebcd-9bf8-43ee-9939-9b6772d76174%40googlegroups.com.

[Eric Longuemare]

Hello,

So thanks @Maks

[Eric Longuemare]

Hello,

I update mod_pagespeed to last beta release this night - 1.11.33.3-0 .

ImageMaxRewritesAtOnce set to -1 gives no image rewrite at all - all of them send a "Too busy to rewrite image"

Set ImageMaxRewritesAtOnce to 0 solve this.

Thanks,

Eric


Le vendredi 27 mai 2016 01:05:12 UTC+2, Eric Longuemare a écrit :

Hello,

So thanks @Maks

[Steve Hill]

Ugh, you're right Eric.There's been a lot of change in that code recently and -1 was fixed in the new code but not properly for the code in 1.11.33. Sorry about that.

[Steve Hill]

Ugh, you're right Eric.There's been a lot of change in that code recently and -1 was fixed in the new code but not properly for the code in 1.11.33. Sorry about that.

--

You received this message because you are subscribed to the Google Groups "mod-pagespeed-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mod-pagespeed-discuss+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/mod-pagespeed-discuss/f21066de-8314-451a-8bd7-da833a628491%40googlegroups.com.