shouldFilterResponse based on response headers?

[Caio Fernando Bertoldi Paes de Andrade]

Hey guys,

I have an issue, again. x)

Actually, I can only decide if response have to be filtered based on request. If I want to decide to filter or not based on response, I need to embed this conditional code into filterResponse. Even if I want to decide only based in headers.

This wouldn't be ugly if filter didn't need all chunks.

I thought in a shouldFilterResponse that gets responses headers (that comes with the first chunk), and that let we decide it based on those headers.

Then, if we want to decide it by content, we still embed it into filterResponse, wait all chunks to be loaded and filter. In this case this make sense.

But I think that for headers only we can improve it. :)

A real example of this issue is when downloading files. If user is downloading an installer from a filtered site, and we want to filter by response, LittleProxy buffers all the exe|msi and only after all chunks received, it applies the filter and then send it to user. This is really ugly because user don't get any feedback from the browser, that still waiting the page|file by entire download time, and then it downloads.

I hope that I could explain my idea. x)

Cheers,

Caio.

[Adam Fisk]

Ahh right -- this one slipped through the cracks. The original thought behind this was that you'd likely want to manipulate an HTML file, for example, before sending it back to the client. In that case, you pretty much need to the whole response first.

So maybe the filter itself should have a flag for whether to filter on the entire response or just the headers, something like that? Seems like sometimes you might want to filter on the whole thing and sometimes not, which gets super tricky given that extra response processors are attached to the pipeline.

Would making the filter configurable for header only versus header plus response body solve the issue for your use case?

-Adam

[caio...@gmail.com]

Adam,

Thank you for your reply. :D

I will explain clearer the problem: I need to modify HTML, of course, and I can do that actually pretty well. The problem is with other resources. Installers, executables, PDFs are being filtered too. They are big, and LittleProxy have to fetch them entirely to check if they are HTML or not, and then pass it to browser, causing a delay visible to proxy users, and in some cases problems in browsing.

I thought that shouldFilterResponses would be called before every response, and supposing that I imagined another implementation of it, receiving by parameter the headers instead of request. But I saw in another thread that it was written that way by design, and now I have no clue how is the best way to solve that delay.

/*

Caio Fernando Bertoldi Paes de Andrade

http://about.me/caiofbpa

*/

[Adam Fisk]

Ahh makes sense -- so if shouldFilterResponses was instead
shouldFilterResponse (without the "s"), that would solve the problem.

I'm thinking something along these lines should do the trick:

http://lists.jboss.org/pipermail/netty-users/2009-March/000346.html

I should be able to insert some sort of shouldFilter callback in the
proxy chain that way, decompressing or not dynamically each time.

I can't knock that out right now, but that's what I'm thinking!

-Adam

[caio...@gmail.com]

Yes, this is exactly what I imagined.

A shouldFilterResponse (without trailing 's') that will let we decide to filter a specific response based on its headers, avoiding extra load.

I guess that change would benefit anyone that uses filters. :)

I will see if I can write something, then if I get some success I'll open a pull request.

Regards,

/*

Caio Fernando Bertoldi Paes de Andrade

http://about.me/caiofbpa

*/

[Adam Fisk]

Sounds great Caio. Let me know, as I'll dig in at some point if you don't get to it and want to make sure we don't duplicate our efforts. Would be a nice change!

[caio...@gmail.com]

Oh, actually I have some overwork, and I guess I have no time to write that, at least until next week.

Cheers,

Caio.

[karan waraich]

Is there any update regarding coz i have the same issue .
i want to get notification from proxy server that it started downloading the file so that we can notify the user on browser.