use exaproxy as an ICAP server for SQUID

[Warren Howard]

Hi,

Is it possible to use exaproxy as a ICAP server for SQUID?

Regards,

Warren.

[Thomas Mangin]

Hello Warren,

It is something we may be doing but for the moment it is not possible.

But if you already use a SQUID redirector, you should be able to use it with ExaProxy without changes.

Thomas

--
You received this message because you are subscribed to the Google Groups "exaproxy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to exaproxy-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Warren Howard]

On Fri, Feb 8, 2013 at 11:58 PM, Thomas Mangin <tho...@mangin.com> wrote:

But if you already use a SQUID redirector, you should be able to use it with ExaProxy without changes.

Dear Thomas,

Thanks for the reply.  I'm trying to do content adaption (I need to add a line to the http header), hence the attempt to use ExaProxy as an icap server.

Regards,

Warren.

[Thomas Mangin]

Warren,

ExaProxy is a content adaption proxy, so you can use it to perform the change.  It can be used to replace SQUID, or simply placed in front of it should you need caching, as you can configure it to send all its request to a backend SQUID server.

The format used by ExaProxy to perform content adaption is compatible with ICAP (it is a subset of so the parsing can be faster than normal ICAP), so you can perform header insertion should you need. This choice was made as we always thought it could be turned into an ICAP server - but we do currently not have the time to start this project.

Regards,

Thomas

[Warren Howard]

Dear Thomas,

To use ExaProxy in place of SQUID to insert a line into the http header I would need to change the exaproxy.redirector.protocol from url to icap:://<uri>, is this correct?  Where uri, could be any path or location, because it's not actually being used as an ICAP server at this stage?

Regards,

Warren.

Phone : +919840518408

Email : war...@meghaconsulting.com

Web : www.meghaconsulting.com

[Thomas Mangin]

Hello Warren,

> To use ExaProxy in place of SQUID to insert a line into the http header I would need to change the exaproxy.redirector.protocol from url to icap:://<uri>, is this correct?
> Where uri, could be any path or location, because it's not actually being used as an ICAP server at this stage?

Nearly correct. The ICAP <uri> is used when generating the ICAP REQMOD (it is the Host part of it), so at this point it does not matter much and can be anything. As the request are not sent to any remote server.

If you run exaproxy this way (from it tarball folder):

env \
exaproxy.redirector.enable=true \
exaproxy.redirector.protocol=icap://example.com \
exaproxy.redirector.program=./etc/exaproxy/redirector/icap-google-to-others \
./sbin/exaproxy

And use chrome for example to go to http://www.google.com you will find that your request have been intercepted to Wolfram|Alpha

I noticed that "exaproxy -h" was broken (a silly one liner bug - sigh !). So I fixed it and release version 1.0.5.
-h gives you information about every exaproxy option.

The documentation for the project is .. lacking .. so feel free to ask as many question as you want.

If you look at : ./etc/exaproxy/redirector/icap-safe-youtube. This does cookie manipulation to for "safe youtube".
You will want to do something similar. read the headers, modify them and return them with the ICAP header.

Thomas

[Warren Howard]

Dear Thomas,

Thanks for the support, I download and installed 1.0.5 made the changes suggested by you.  I can browse the net via Exaproxy and the redirection to wolframalpha.com worked once but most of the time gives :

" WEBSITE NOT FOUND

www.wolframalpha.com

could not be found on the internet, are you sure that the domain name is correct ?"

I also loaded up Safe Youtube, at first all pages were giving "WEBSITE NOT FOUND", but after reloading youtube.com three or more times the page started to work and safe youtube would remain always on.  Other pages started to work too, but to get www.google.com to work I needed to reloaded it at least 3 times.

It looks as though there is some fine tuning required --- where do you suggest I start?  Also, I went back and tried icap-google-to-others again and following the same reload three or more (sometimes up to 8) times technique and was able to get www.wolframalpha.com to load when visiting www.google.com.  One more observation, https://www.google.com goes to Google and is not redirected, is this due to difficulties in doing content adaptation on encrypted pages or simply because the redirector has not been written to handle https pages?

Regards,

Warren.

Phone : +919840518408

Email : war...@meghaconsulting.com

Web : www.meghaconsulting.com

[Thomas Mangin]

On 11 Feb 2013, at 09:05, Warren Howard <war...@meghaconsulting.com> wrote:

Dear Thomas,

Thanks for the support, I download and installed 1.0.5 made the changes suggested by you.  I can browse the net via Exaproxy and the redirection to wolframalpha.com worked once but most of the time gives :

" WEBSITE NOT FOUND

www.wolframalpha.com

could not be found on the internet, are you sure that the domain name is correct ?"

I also loaded up Safe Youtube, at first all pages were giving "WEBSITE NOT FOUND", but after reloading youtube.com three or more times the page started to work and safe youtube would remain always on.  Other pages started to work too, but to get www.google.com to work I needed to reloaded it at least 3 times.

It looks like a DNS related thing.

It could be :

 - a bug in the expiry of the DNS record

 - an unreliable DNS and we do not cope well with it - we always have a locally installed DNS, so it would never happen in our setup. I will ask David if we send the request more than once, which could get lost on unreliable connections.

 - something else ..

That said I can not reproduce this issue on my MacBook, but we will investigate this and let you know how it goes.

It looks as though there is some fine tuning required --- where do you suggest I start?  

Install a local DNS resolver on your computer ( unbound for example ), and change your nameserver in /etc/resolv.conf to 127.0.0.1

Also, I went back and tried icap-google-to-others again and following the same reload three or more (sometimes up to 8) times technique and was able to get www.wolframalpha.com to load when visiting www.google.com.  

This again looks like DNS related and the same issue as you describe before.

One more observation, https://www.google.com goes to Google and is not redirected, is this due to difficulties in doing content adaptation on encrypted pages or simply because the redirector has not been written to handle https pages?

You should read what the redirector program does, it does not deal with HTTPS.

We can only see the Host part of the CONNECT message (the headers, cookie, etc. are all encrypted) so it is only possible to perform host based actions (which may be an IP and not the site name too).

Normally, you can not intercept https traffic, the only reason we can cheat with google is their nossl site which allow to recover the connection via HTTP using nosslsearch.google.com.

An example of it is in icap-google-no-ssl

Thomas