ngx_pagespeed - using tcpinfo.rtt as an input for tuning options

[Otto van der Schaaf]

Nginx exposes a couple of variables [1] when tcpinfo is available on a system.

These variables may be worth looking into as they might provide input for tuning the options at request-time.

Two examples that I can come up with are:

- The rewrite deadline, when rount trip times are large it may make sense to spend more time optimizing before sending a response.

- Switch to OptimizeForBandwidth/scale image compression settings 

But some expirimentation is needed to determine if/how this works in practice.

The single barrier for people to experiment with this right now is that the suggested settings above are not "scriptable" (like LoadFromFile) [2].

If we could do that, the rest would probably just boil down to experimentation and documentation.

So now I'm wondering what would be against opening up scripting to all directory-scoped options?

One thing is that the document should state that people need to constrain the inputs for security reasons (which it already does for LoadFromFile).

Does anyone have any other objections or ideas?

[1] $tcpinfo_rtt, $tcpinfo_rttvar, $tcpinfo_snd_cwnd, $tcpinfo_rcv_space

[2] https://developers.google.com/speed/pagespeed/module/domains#LoadFromFileScriptVariables

Otto

[Jeff Kaufman]

Generally in favor of opening up more things for scripting.

Any directory-scoped options that someone could plausibly put a "$"
would make opening them up a non-backwards compatible change. We
could add a different config option, where people would list which
commands they wanted to allow scripting in? But that's pretty
heavyweight.

> --
> You received this message because you are subscribed to the Google Groups
> "pagespeed-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to pagespeed-de...@googlegroups.com.
> To post to this group, send email to pagesp...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pagespeed-dev/CAHqmWiNLJwZZ-CbBw_JeDR0Y2i54bL0ENaDRJQQj72%3DEdP5aSw%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

[Otto van der Schaaf]

I see, yes. Backwards compatibility would be an issue to those that already have "pagespeed ProcessScriptVariables on;"  in their configuration.

No beauty price, but how about starting to interpret the right hand argument (currently always "on"):

- "on": means the current set is allowed, we keep this just for backwards compatibility purposes and remove this from the docs.

- "all": gets you the new behaviour, and allows scripting in any directory-scoped option.

[Jeff Kaufman]

That's a neat idea; I like it. We'd document "all", with a note about
"on" for previous versions that don't support "all" and systems
already configured with "on".

Any objections?

[Otto van der Schaaf]

Sounds good to me, I staged a draft here: https://github.com/pagespeed/ngx_pagespeed/compare/trunk-tracking...oschaaf-trunk-tracking-script-all-directory-options?expand=1#diff-99362d7e68d978dd9356e67022a4712fR281

One thing I am wondering about, is that some options that are specified as server-scoped (like LoadFromFile) are currently subject to processing script variables. The change above marks all query- and directory- scoped options as candidates for containing script variables. 

But are there more options we should include that are not directory- or query- scoped? Is there an extensive list of options around that lists their name and scope? 

It may be important to get that right so we don't have to worry about backward compatibility again later on..

Otto

[Jeff Kaufman]

The full list of options and their scopes would be in the various
rewrite_options.cc AddFooProperty statements.

On Tue, Jul 26, 2016 at 10:46 AM, Otto van der Schaaf

[Otto van der Schaaf]

I logged everything that passes through AddProperty on startup, which gave the following list:

https://docs.google.com/spreadsheets/d/1qJSUZqd2Te-4SDtm1Av6u-AckNv_oHEp1qdaUjNqZoM/edit#gid=0

The green options would be allowed for scripting.

Some options like LoadFromFileXXX, MapRewriteDomain and MapOriginDomain do not seem to pass through AddBaseProperty. 

Digging some more, it looks like RewriteOptionsTest::LookupNonBaseOptionByNameTest contains another bunch of options I need to check.