Help with never_direct nonhierarchical_direct

[Eduardo Meyer]

Hello dear gents,

I have changed my strategy recently, deploying Lusca for filtering and
something else for caching, what I would like to have is the following
behavior:

- Lusca does it best and caches everything it can
- Non-cacheable objects should be redirected to a cache peer

I have tried with never_direct on and what happens is that everything
goes to the cache peer, including what Lusca could handle itself;
I have tried with nonhierarchical_direct off which by description on
squid.conf is exactly what I need, but absolutely everything goes to
FIRST_UP_PARENT too, nothing gets handled by Lusca :-(

My current cache_peer config is:

cache_peer 10.10.2.10 parent 8080 0 proxy-only no-digest no-query
dead_peer_timeout 2 seconds

I dont have a cache_peer_access directive in place.

I would love to be able to use something like:

cache_peer_access 10.10.2.10 <a fallback match acl for DIRECT>

For example shouldn't it work:

cache_peer 10.10.2.10 parent 8080 0 proxy-only no-digest no-query
dead_peer_timeout 2 seconds
acl test hier_code DIRECT
cache_peer_access 10.10.2.10 allow test

Should it work? because it didnt :-( Everything goes DIRECT without
going to cache peer...

Any suggestions are very, very welcome :-)

Now to the hackers: if I need to directly change it in source code,
where should I? I mean where, which source code file/line the
go-DIRECT decision takes place? Thank you :-)

--
===========
Eduardo Meyer
pessoal: dudu....@gmail.com
profissional: ddm.fa...@saude.gov.br

[H]

Eduardo Meyer wrote:
> Hello dear gents,
>
> I have changed my strategy recently, deploying Lusca for filtering and
> something else for caching, what I would like to have is the following
> behavior:
>
> - Lusca does it best and caches everything it can
> - Non-cacheable objects should be redirected to a cache peer
>

man ... makes not so much sense to me

you can not redirect objects, but if you meant to say trying to fetch
none-cacheable objects from peer, what sense does it make fetching
none-cacheable objects from a cache server?

anyway ... :) not anything makes sense in life, so look if it helps

>
> My current cache_peer config is:
>
> cache_peer 10.10.2.10 parent 8080 0 proxy-only no-digest no-query
> dead_peer_timeout 2 seconds
>

this should be enough:

cache_peer 10.10.2.10 parent 8080 0

this way you get it from whom serves first, direct or cache_peer

in your case, proxy-only may not make sense since your cache can not
know if the something-else-cache serves from cache or not, whatsoever,
it makes no sense since you are already fetching none-cacheable objects,
so why do you care?

setting icp port to 0 already implies no-query

in order not to query cache_peer for everything you need to define your
"none-cacheable" objects/taregts and write your acl, then

cache_peer_access cache_peer_IP deny ! ACL_NAME

this is pretty much what you need for trying to fetch your
none-cacheable objects from a cache peer :)

Hans

> I dont have a cache_peer_access directive in place.
>
> I would love to be able to use something like:
>
> cache_peer_access 10.10.2.10 <a fallback match acl for DIRECT>
>
> For example shouldn't it work:
>
> cache_peer 10.10.2.10 parent 8080 0 proxy-only no-digest no-query
> dead_peer_timeout 2 seconds
> acl test hier_code DIRECT
> cache_peer_access 10.10.2.10 allow test
>
> Should it work? because it didnt :-( Everything goes DIRECT without
> going to cache peer...
>
> Any suggestions are very, very welcome :-)
>
> Now to the hackers: if I need to directly change it in source code,
> where should I? I mean where, which source code file/line the
> go-DIRECT decision takes place? Thank you :-)
>


--

H
+55 11 4249.2222

[Eduardo Meyer]

Dear Hans,

Thanks for your reply, in first place.

On Tue, May 8, 2012 at 12:53 AM, H <h...@hm.net.br> wrote:
> Eduardo Meyer wrote:
>> Hello dear gents,
>>
>> I have changed my strategy recently, deploying Lusca for filtering and
>> something else for caching, what I would like to have is the following
>> behavior:
>>
>> - Lusca does it best and caches everything it can
>> - Non-cacheable objects should be redirected to a cache peer
>>
>
> man ... makes not so much sense to me
>
> you can not redirect objects, but if you meant to say trying to fetch
> none-cacheable objects from peer, what sense does it make fetching
> none-cacheable objects from a cache server?

Non cacheable objects from the Lusca perspective. Possibly cacheable
from the upper peer which is not Lusca. In fact I am using Lusca to do
some filtering, user auth and primary caching, and peering it with
something else to do other kind of caching which Lusca can't handle by
itself.

>
> anyway ... :) not anything makes sense in life, so look if it helps
>
>
>>
>> My current cache_peer config is:
>>
>> cache_peer 10.10.2.10 parent 8080 0 proxy-only no-digest no-query
>> dead_peer_timeout 2 seconds
>>
>
> this should be enough:
>
> cache_peer 10.10.2.10 parent 8080 0
>
> this way you get it from whom serves first, direct or cache_peer
>
> in your case, proxy-only may not make sense since your cache can not
> know if the something-else-cache serves from cache or not, whatsoever,
> it makes no sense since you are already fetching none-cacheable objects,
> so why do you care?

I have tried, and still, Lusca goes to upper peer for everything,
including the objects it *can* handle. I want to say "Hey Lusca, do
your best and cache everything you can, the objects you can't handle,
use the upper cache peer instead of going direct".

I want the upper peer to decide to go direct, not Lusca.

> setting icp port to 0 already implies no-query
>
> in order not to query cache_peer for everything you need to define your
> "none-cacheable" objects/taregts and write your acl, then
>
> cache_peer_access cache_peer_IP deny ! ACL_NAME

OK, it's the desired approach, but I need to figure out which ones are
Lusca's cacheable objects. Which ones are they? It's not something
that I have in mind, that I have decided, it's something Lusca can or
cannot handle which I don't know how an ACL can match that.

>
> this is pretty much what you need for trying to fetch your
> none-cacheable objects from a cache peer :)
>
> Hans
>
>
>> I dont have a cache_peer_access directive in place.
>>
>> I would love to be able to use something like:
>>
>> cache_peer_access 10.10.2.10 <a fallback match acl for DIRECT>
>>
>> For example shouldn't it work:
>>
>> cache_peer 10.10.2.10 parent 8080 0 proxy-only no-digest no-query
>> dead_peer_timeout 2 seconds
>> acl test hier_code DIRECT
>> cache_peer_access 10.10.2.10 allow test
>>
>> Should it work? because it didnt :-( Everything goes DIRECT without
>> going to cache peer...
>>
>> Any suggestions are very, very welcome :-)
>>
>> Now to the hackers: if I need to directly change it in source code,
>> where should I? I mean where, which source code file/line the
>> go-DIRECT decision takes place? Thank you :-)
>>
>
>
> --
> H
> +55 11 4249.2222
>

[Eduardo Meyer]

Now, asking myself if what I want makes any sense from Lusca
perspective, I ask you guys:

Does Lusca know what it can cache before actually trying?

Because what I want is to teel lusca to do it's best and cache
everything it can handle, and go to upper cache peer for things it
cannot handle. But if Lusca does not have the object in disk already,
can it know if that required object is cacheable by Lusca itself?

Say, "wow, static object, I can handle it", versus "eeerh weird stuff
via http protocol, I have no idea how to handle it, but instead of
going direct let me redirect to upper peer".

Is it anyhow possible?

[H]

Eduardo Meyer wrote:
> Now, asking myself if what I want makes any sense from Lusca
> perspective, I ask you guys:
>
> Does Lusca know what it can cache before actually trying?
>
> Because what I want is to teel lusca to do it's best and cache
> everything it can handle, and go to upper cache peer for things it
> cannot handle. But if Lusca does not have the object in disk already,
> can it know if that required object is cacheable by Lusca itself?
>
> Say, "wow, static object, I can handle it", versus "eeerh weird stuff
> via http protocol, I have no idea how to handle it, but instead of
> going direct let me redirect to upper peer".
>

you are trying to complicate something pretty simple

theoretically can be cached everything what has an URL associated,
dynamic or static

nevertheless, the content provider can set if he want his content to be
cached or not in the HEAD section

even so, you can violate this rules, it is up to you, everything you
need is in the manuals and squid.conf.default

further you could workaround content provider no-cache-tricks with your
own anti-no-cache-tricks handled by external rewrite programs

your other point, questioning the cache_peer_access scenario, is also as
it is and already explained,

if you can not define what you want to fetch from the peer, well, your
problem, then work without cache_peer_access rules and let squid do by
itself what it does ... caching :)


Hans

H
+55 11 4249.2222

[Eduardo Meyer]

On Tue, May 8, 2012 at 2:38 PM, H <h...@hm.net.br> wrote:
> Eduardo Meyer wrote:
>> Now, asking myself if what I want makes any sense from Lusca
>> perspective, I ask you guys:
>>
>> Does Lusca know what it can cache before actually trying?
>>
>> Because what I want is to teel lusca to do it's best and cache
>> everything it can handle, and go to upper cache peer for things it
>> cannot handle. But if Lusca does not have the object in disk already,
>> can it know if that required object is cacheable by Lusca itself?
>>
>> Say, "wow, static object, I can handle it", versus "eeerh weird stuff
>> via http protocol, I have no idea how to handle it, but instead of
>> going direct let me redirect to upper peer".
>>
>
> you are trying to complicate something pretty simple
>

I'm sorry but my problem, which I seek for a solution, doesn't seem
simple to me at all.

> theoretically can be cached everything what has an URL associated,
> dynamic or static

A theory is a theory, at this time this one theory just doesn't fit my
scenario. I cant just simplify thing that much.

>
> nevertheless, the content provider can set if he want his content to be
> cached or not in the HEAD section
>
> even so, you can violate this rules, it is up to you, everything you
> need is in the manuals and squid.conf.default

I dont care about provider-side caching controls, I dont need to
violate it, it's not the problem :)

>
> further you could workaround content provider no-cache-tricks with your
> own anti-no-cache-tricks handled by external rewrite programs
>
> your other point, questioning the cache_peer_access scenario, is also as
> it is and already explained,

Yeh, it doesn't help much.

> if you can not define what you want to fetch from the peer, well, your
> problem, then work without cache_peer_access rules and let squid do by
> itself what it does ... caching :)

I wish, but Lusca can't cache what I need. They are pretty much
specific traffic and they dont have an URL associated with. They are
stuff like video streaming, bulk updates from server farms and so on.

Thanks for your effort, but I still need for some help if someone else
has the time.

[H]

Eduardo Meyer wrote:
>
> I wish, but Lusca can't cache what I need. They are pretty much
> specific traffic and they dont have an URL associated with. They are
> stuff like video streaming, bulk updates from server farms and so on.
>

so now we are talking ...
now you say you know what you want

there is just running a thread on squid-users about new rewriting helper
adapting to the new youtube range policy, windows update and other stuff

look for subject "anyone knows some info about youtube range parameter"

[Eduardo Meyer]

Im following that thread but wont help me too.

I dont care about youtube, and general tube videos; Im not an ISP;
what I need is not related to a specific website, technology, I gave
you some valid examples, but they are pretty much more rtsp video
streaming, quicktime server stuff, sometimes a kind of H323, and a
special kind of database binary update somehow similar to portsnap
deltas, but with unpredictable URLs, file format and stuff which most
them CAN be cached by a very specific caching system, while I need
Lusca to still take care of the web content;

Nowadays http and tc/80 are not synonym for "the web" itsel; while
Lusca is good at caching the web, I want the rest to be treated by
another peer.

Thank you again.

>
>
>
> --
> H
> +55 11 4249.2222
>