Lusca-1.0 released

22 views
Skip to first unread message

Adrian Chadd

unread,
Feb 2, 2009, 5:58:32 PM2/2/09
to lusca...@googlegroups.com
I've just uploaded the first Lusca-1.0 release tarball.

You can grab it here: http://code.google.com/p/lusca-cache/downloads/list

Its effectively Cacheboy-1.6 with a namechange and a couple of DoS
bugs fixed. I'll port those bug fixes to Cacheboy-1.6 and re-roll the
freebsd package tonight or tomorrow, but please, if you're using
Cacheboy-1.*, I do suggest you upgrade to this lusca tarball as soon
as possible.

Thanks!


Adrian

Faysal Banna

unread,
Feb 3, 2009, 10:30:17 AM2/3/09
to lusca...@googlegroups.com
I have installed Lusca now and giving it a try
one question
does it support TProxy4 ?????
regards
--
============================
        Faysal Banna
Meteorological Services
Rafic Harriri International Airport
     Beirut - Lebanon
   Mob: +961-3-258043
=============================

Faysal Banna

unread,
Feb 4, 2009, 8:25:03 AM2/4/09
to lusca...@googlegroups.com
As i said before this is some statistics to what the cache is performing ....

nevertheless why the dns lookups is zero ?????

by the way before i had squid-3.1 but for god's sake it was awful in performance comparing to this one .....

maybe because of the TProxy4.1 patch but this is still questionable
i was unable to make lots of benchmarks because i needed to get this running asap ...
in this info i provide i have to clear out its handling something around 19Mbit/s yet on peak hours which is couple of hours from now i shall post the same info just for comparison where my peak would go for something around 30Mbit/s
How can i enhance the responsivness of near hits ????


Regards

HTTP/1.0 200 OK
Server: Lusca/LUSCA_1.0
Date: Wed, 04 Feb 2009 13:17:30 GMT
Content-Type: text/plain
Expires: Wed, 04 Feb 2009 13:17:30 GMT
X-Cache: MISS from UbiOne-t1
X-Cache-Lookup: MISS from UbiOne-t1:33128
Connection: close

Squid Object Cache: Version LUSCA_1.0
Start Time:    Tue, 03 Feb 2009 15:31:27 GMT
Current Time:    Wed, 04 Feb 2009 13:17:30 GMT
Connection information for squid:
    Number of clients accessing cache:    21
    Number of HTTP requests received:    9282742
    Number of ICP messages received:    0
    Number of ICP messages sent:    0
    Number of queued ICP replies:    0
    Request failure ratio:     0.00
    Average HTTP requests per minute since start:    7107.4
    Average ICP messages per minute since start:    0.0
    Select loop called: 149550027 times, 0.524 ms avg
Cache information for squid:
    Request Hit Ratios:    5min: 26.7%, 60min: 28.2%
    Byte Hit Ratios:    5min: 23.6%, 60min: 19.8%
    Request Memory Hit Ratios:    5min: 16.3%, 60min: 14.6%
    Request Disk Hit Ratios:    5min: 42.3%, 60min: 39.5%
    Storage Swap size:    899408356 KB
    Storage Mem size:    1535876 KB
    Mean Object Size:    27.40 KB
    Requests given to unlinkd:    0
Median Service Times (seconds)  5 min    60 min:
    HTTP Requests (All):   0.35832  0.33943
    Cache Misses:          0.42149  0.42149
    Cache Hits:            0.01309  0.01235
    Near Hits:             0.33943  0.30459
    Not-Modified Replies:  0.00000  0.00000
    DNS Lookups:           0.00000  0.00000
    ICP Queries:           0.00000  0.00000
Resource usage for squid:
    UP Time:    78363.497 seconds
    CPU Time:    9532.360 seconds
    CPU Usage:    12.16%
    CPU Usage, 5 minute avg:    14.96%
    CPU Usage, 60 minute avg:    14.80%
    Process Data Segment Size via sbrk(): 1025132 KB
    Maximum Resident Size: 0 KB
    Page faults with physical i/o: 25962
Memory usage for squid via mallinfo():
    Total space in arena:  -1100692 KB
    Ordinary blocks:       -1103011 KB   1121 blks
    Small blocks:               0 KB      0 blks
    Holding blocks:        462388 KB     15 blks
    Free Small blocks:          0 KB
    Free Ordinary blocks:    2319 KB
    Total in use:          -640623 KB 100%
    Total free:              2319 KB 0%
    Total size:            -638304 KB
Memory accounted for:
    Total accounted:       5282232 KB
    memPoolAlloc calls: 1561769830
    memPoolFree calls: 1489089946
File descriptor usage for squid:
    Maximum number of file descriptors:   64000
    Largest file desc currently in use:   1675
    Number of file desc currently in use: 1272
    Files queued for open:                   0
    Available number of file descriptors: 62728
    Reserved number of file descriptors:   100
    Store Disk files open:                  83
    IO loop method:                     epoll
Internal Data Structures:
    32866548 StoreEntries
    294883 StoreEntries with MemObjects
    294274 Hot Object Cache Items
    32824740 on-disk objects

Asif Bakali

unread,
Feb 4, 2009, 9:04:44 AM2/4/09
to lusca...@googlegroups.com
can  you please send the bugs fixed list ..

Adrian Chadd

unread,
Feb 4, 2009, 11:32:53 AM2/4/09
to lusca...@googlegroups.com
2009/2/4 Faysal Banna <degr...@gmail.com>:

> As i said before this is some statistics to what the cache is performing
> ....

Hm!

> nevertheless why the dns lookups is zero ?????

No idea. They shouldn't be zero. But then, I did move around the dns
code into a separate library and its possible I didn't link the
statistics reporting bits in.

Tell me exactly where you're seeing 0 dns lookups in the statistics
and I'll go off and fix it.

> by the way before i had squid-3.1 but for god's sake it was awful in
> performance comparing to this one .....

:) Thats exactly what I like to hear.

> maybe because of the TProxy4.1 patch but this is still questionable
> i was unable to make lots of benchmarks because i needed to get this running
> asap ...

Live, production testing is fine. :)

> in this info i provide i have to clear out its handling something around
> 19Mbit/s yet on peak hours which is couple of hours from now i shall post
> the same info just for comparison where my peak would go for something
> around 30Mbit/s

:)

> How can i enhance the responsivness of near hits ????

> Median Service Times (seconds) 5 min 60 min:


> HTTP Requests (All): 0.35832 0.33943
> Cache Misses: 0.42149 0.42149
> Cache Hits: 0.01309 0.01235
> Near Hits: 0.33943 0.30459

Whats wrong with 0.3sec near hits? :)

Adrian

Adrian Chadd

unread,
Feb 4, 2009, 11:33:47 AM2/4/09
to lusca...@googlegroups.com
There isn't a "bugs fixed" list right now. I'm going to make much
better use of the issue tracking system so I don't have to generate
"bugs fixed" lists.

I'll make sure I put the security exploit one into an issue.


adrian

2009/2/4 Asif Bakali <asifb...@gmail.com>:

Faysal Banna

unread,
Feb 4, 2009, 11:57:09 AM2/4/09
to lusca...@googlegroups.com
Adrian
check out the timings for the DNS
Not-Modified Replies:  0.00000  0.00000
    DNS Lookups:           0.00000  0.00000
    ICP Queries:           0.00000  0.00000
Regards

By the way is there any near future for incuding TProxy4.1 ???




Regards

Faysal Banna

unread,
Feb 4, 2009, 12:06:03 PM2/4/09
to lusca...@googlegroups.com
Here is my evening statistics where my bandwidth is almost consumed .... ~30Mbit/s

with almost 3000 Users online

By the way Adrian ... can u give some hints that you know to tune the system for optimum performance ..... a collaboration between responsiveness and maximum caching ?????

Regards

squidclient  -p 33128 mgr:info

HTTP/1.0 200 OK
Server: Lusca/LUSCA_1.0
Date: Wed, 04 Feb 2009 17:02:14 GMT
Content-Type: text/plain
Expires: Wed, 04 Feb 2009 17:02:14 GMT

X-Cache: MISS from UbiOne-t1
X-Cache-Lookup: MISS from UbiOne-t1:33128
Connection: close

Squid Object Cache: Version LUSCA_1.0
Start Time:    Tue, 03 Feb 2009 15:31:27 GMT
Current Time:    Wed, 04 Feb 2009 17:02:14 GMT
Connection information for squid:
    Number of clients accessing cache:    18
    Number of HTTP requests received:    12133682

    Number of ICP messages received:    0
    Number of ICP messages sent:    0
    Number of queued ICP replies:    0
    Request failure ratio:     0.00
    Average HTTP requests per minute since start:    7926.5

    Average ICP messages per minute since start:    0.0
    Select loop called: 190267968 times, 0.483 ms avg
Cache information for squid:
    Request Hit Ratios:    5min: 23.5%, 60min: 25.0%
    Byte Hit Ratios:    5min: 19.6%, 60min: 18.8%
    Request Memory Hit Ratios:    5min: 15.5%, 60min: 16.1%
    Request Disk Hit Ratios:    5min: 35.8%, 60min: 33.6%
    Storage Swap size:    908469352 KB
    Storage Mem size:    1535832 KB
    Mean Object Size:    27.38 KB

    Requests given to unlinkd:    0
Median Service Times (seconds)  5 min    60 min:
    HTTP Requests (All):   0.44492  0.42149
    Cache Misses:          0.46965  0.46965
    Cache Hits:            0.01648  0.01235
    Near Hits:             0.42149  0.37825

    Not-Modified Replies:  0.00000  0.00000
    DNS Lookups:           0.00000  0.00000
    ICP Queries:           0.00000  0.00000
Resource usage for squid:
    UP Time:    91846.893 seconds
    CPU Time:    12227.770 seconds
    CPU Usage:    13.31%
    CPU Usage, 5 minute avg:    22.21%
    CPU Usage, 60 minute avg:    22.86%

    Process Data Segment Size via sbrk(): 1025132 KB
    Maximum Resident Size: 0 KB
    Page faults with physical i/o: 26855

Memory usage for squid via mallinfo():
    Total space in arena:  -994196 KB
    Ordinary blocks:       -1002977 KB   3535 blks

    Small blocks:               0 KB      0 blks
    Holding blocks:        462388 KB     15 blks
    Free Small blocks:          0 KB
    Free Ordinary blocks:    8780 KB
    Total in use:          -540589 KB 102%
    Total free:              8780 KB -1%
    Total size:            -531808 KB
Memory accounted for:
    Total accounted:       5348604 KB
    memPoolAlloc calls: 1989411385
    memPoolFree calls: 1915675896

File descriptor usage for squid:
    Maximum number of file descriptors:   64000
    Largest file desc currently in use:   2499
    Number of file desc currently in use: 2252

    Files queued for open:                   0
    Available number of file descriptors: 61748

    Reserved number of file descriptors:   100
    Store Disk files open:                 160

    IO loop method:                     epoll
Internal Data Structures:
    33238535 StoreEntries
    302007 StoreEntries with MemObjects
    300932 Hot Object Cache Items
    33185983 on-disk objects

Adrian Chadd

unread,
Feb 4, 2009, 12:19:05 PM2/4/09
to lusca...@googlegroups.com
There should be basic tproxy-4 support in lusca-1.0 (like it is in
cacheboy-1.6). ./configure --help | grep tproxy



Adrian

2009/2/4 Faysal Banna <degr...@gmail.com>:

Adrian Chadd

unread,
Feb 4, 2009, 12:19:44 PM2/4/09
to lusca...@googlegroups.com
Also, please add the "no dns stats" in as a Lusca issue:

http://code.google.com/p/lusca-cache/issues/list

Thanks!


Adrian

2009/2/4 Faysal Banna <degr...@gmail.com>:

Asif Bakali

unread,
Feb 4, 2009, 2:15:51 PM2/4/09
to lusca...@googlegroups.com
current i am running 1.6 and here is my result can you recomended to replace with lusca ? we are running 15 Mbit/s
 
HTTP/1.0 200 OK
Server: Cacheboy/CACHEBOY_1.6
Date: Wed, 04 Feb 2009 19:02:39 GMT
Content-Type: text/plain
Expires: Wed, 04 Feb 2009 19:02:39 GMT
X-Cache: MISS from linto
Connection: close
Squid Object Cache: Version CACHEBOY_1.6
Start Time:     Sun, 01 Feb 2009 08:38:26 GMT
Current Time:   Wed, 04 Feb 2009 19:02:39 GMT
Connection information for squid:
        Number of clients accessing cache:      0
        Number of HTTP requests received:       18081781
        Number of ICP messages received:        2

        Number of ICP messages sent:    0
        Number of queued ICP replies:   0
        Request failure ratio:   0.00
        Average HTTP requests per minute since start:   3657.2

        Average ICP messages per minute since start:    0.0
        Select loop called: 306076035 times, 0.969 ms avg
Cache information for squid:
        Request Hit Ratios:     5min: 36.0%, 60min: 41.0%
        Byte Hit Ratios:        5min: 21.9%, 60min: 25.2%
        Request Memory Hit Ratios:      5min: 10.5%, 60min: 11.2%
        Request Disk Hit Ratios:        5min: 56.4%, 60min: 56.4%
        Storage Swap size:      60375512 KB
        Storage Mem size:       131088 KB
        Mean Object Size:       20.04 KB

        Requests given to unlinkd:      0
Median Service Times (seconds)  5 min    60 min:
        HTTP Requests (All):   0.49576  0.44492
        Cache Misses:          0.94847  0.85130
        Cache Hits:            0.01469  0.02069
        Near Hits:             0.55240  0.58309
        Not-Modified Replies:  0.00091  0.00179

        DNS Lookups:           0.00000  0.00000
        ICP Queries:           0.00000  0.00000
Resource usage for squid:
        UP Time:        296653.328 seconds
        CPU Time:       38514.459 seconds
        CPU Usage:      12.98%
        CPU Usage, 5 minute avg:        25.57%
        CPU Usage, 60 minute avg:       25.75%
        Process Data Segment Size via sbrk(): 536436 KB

        Maximum Resident Size: 0 KB
        Page faults with physical i/o: 2
Memory usage for squid via mallinfo():
        Total space in arena:  536436 KB
        Ordinary blocks:       519261 KB 923432 blks

        Small blocks:               0 KB      0 blks
        Holding blocks:         54568 KB      8 blks

        Free Small blocks:          0 KB
        Free Ordinary blocks:   17174 KB
        Total in use:          573829 KB 97%
        Total free:             17174 KB 3%
        Total size:            591004 KB
Memory accounted for:
        Total accounted:       363942 KB
        memPoolAlloc calls: 2313198608
        memPoolFree calls: 2306453294

File descriptor usage for squid:
        Maximum number of file descriptors:   65536
        Largest file desc currently in use:   1702
        Number of file desc currently in use: 1430
        Files queued for open:                   1
        Available number of file descriptors: 64105

        Reserved number of file descriptors:   100
        Store Disk files open:                 120

        IO loop method:                     epoll
Internal Data Structures:
        3036357 StoreEntries
         29777 StoreEntries with MemObjects
         28563 Hot Object Cache Items
        3012797 on-disk objects
###################################################

Faysal Banna

unread,
Feb 4, 2009, 5:00:35 PM2/4/09
to lusca...@googlegroups.com
This is another stat ....
behold that its capable of something of around 140 req/sec
but when request goes over this limit  i start to see degradation in performance ....
by the way what are those errors i see in my cache.log as performance starts to degrade drastically ....?

2009/02/04 19:51:57| storeUpdateCopy: Aborted at 372225 (0)
2009/02/04 20:01:33| storeUpdateCopy: Aborted on write at 91521 (2687)
2009/02/04 20:38:39| storeUpdateCopy: Aborted at 313147 (0)
2009/02/04 20:38:39| storeUpdateCopy: Aborted at 313147 (0)
2009/02/04 20:38:39| storeUpdateCopy: Aborted at 313147 (0)
2009/02/04 20:38:39| storeUpdateCopy: Aborted at 313147 (0)
2009/02/04 20:38:39| storeUpdateCopy: Aborted at 313147 (0)
2009/02/04 20:38:39| storeUpdateCopy: Aborted at 313147 (0)
2009/02/04 21:24:57| storeUpdateCopy: Aborted at 64735 (0)
2009/02/04 22:31:30| storeUpdateCopy: Error at 338 (-1)
2009/02/04 22:31:30| storeUpdateCopy: Error at 293 (-1)
2009/02/04 22:31:30| storeUpdateCopy: Error at 293 (-1)
2009/02/04 22:31:30| storeUpdateCopy: Error at 213 (-1)
2009/02/04 22:31:31| storeUpdateCopy: Error at 424 (-1)
2009/02/04 22:31:31| storeUpdateCopy: Error at 340 (-1)
2009/02/04 22:31:33| storeUpdateCopy: Error at 272 (-1)
2009/02/04 22:31:41| storeUpdateCopy: Error at 267 (-1)
2009/02/04 22:31:50| storeUpdateCopy: Error at 322 (-1)
2009/02/04 22:31:50| storeUpdateCopy: Error at 298 (-1)
2009/02/04 22:31:50| storeUpdateCopy: Error at 236 (-1)



squidclient  -p 33128 mgr:info

HTTP/1.0 200 OK
Server: Lusca/LUSCA_1.0
Date: Wed, 04 Feb 2009 21:54:33 GMT
Content-Type: text/plain
Expires: Wed, 04 Feb 2009 21:54:33 GMT

X-Cache: MISS from UbiOne-t1
X-Cache-Lookup: MISS from UbiOne-t1:33128
Connection: close

Squid Object Cache: Version LUSCA_1.0
Start Time:     Wed, 04 Feb 2009 20:40:14 GMT
Current Time:   Wed, 04 Feb 2009 21:54:33 GMT
Connection information for squid:
        Number of clients accessing cache:      16
        Number of HTTP requests received:       629774
        Number of ICP messages received:        0

        Number of ICP messages sent:    0
        Number of queued ICP replies:   0
        Request failure ratio:   0.00
        Average HTTP requests per minute since start:   8474.1

        Average ICP messages per minute since start:    0.0
        Select loop called: 9473781 times, 0.471 ms avg
Cache information for squid:
        Request Hit Ratios:     5min: 18.8%, 60min: 18.8%
        Byte Hit Ratios:        5min: 16.3%, 60min: 16.0%
        Request Memory Hit Ratios:      5min: 11.9%, 60min: 7.3%
        Request Disk Hit Ratios:        5min: 41.2%, 60min: 42.9%
        Storage Swap size:      917500980 KB
        Storage Mem size:       1288700 KB
        Mean Object Size:       27.32 KB

        Requests given to unlinkd:      0
Median Service Times (seconds)  5 min    60 min:
        HTTP Requests (All):   0.44492  0.46965
        Cache Misses:          0.46965  0.52331
        Cache Hits:            0.00379  0.02069
        Near Hits:             0.42149  0.46965
        Not-Modified Replies:  0.00091  0.00091

        DNS Lookups:           0.00000  0.00000
        ICP Queries:           0.00000  0.00000
Resource usage for squid:
        UP Time:        4459.074 seconds
        CPU Time:       893.410 seconds
        CPU Usage:      20.04%
        CPU Usage, 5 minute avg:        14.01%
        CPU Usage, 60 minute avg:       15.21%
        Process Data Segment Size via sbrk(): 1038632 KB

        Maximum Resident Size: 0 KB
        Page faults with physical i/o: 0

Memory usage for squid via mallinfo():
        Total space in arena:  -1610456 KB
        Ordinary blocks:       -1611861 KB    786 blks

        Small blocks:               0 KB      0 blks
        Holding blocks:        460136 KB     15 blks

        Free Small blocks:          0 KB
        Free Ordinary blocks:    1404 KB
        Total in use:          -1151725 KB 100%
        Total free:              1404 KB 0%
        Total size:            -1150320 KB
Memory accounted for:
        Total accounted:       4860084 KB
        memPoolAlloc calls: 232579002
        memPoolFree calls: 162370863

File descriptor usage for squid:
        Maximum number of file descriptors:   64000
        Largest file desc currently in use:   1699
        Number of file desc currently in use: 1355

        Files queued for open:                   0
        Available number of file descriptors: 62645

        Reserved number of file descriptors:   100
        Store Disk files open:                  89

        IO loop method:                     epoll
Internal Data Structures:
        33591953 StoreEntries
        117164 StoreEntries with MemObjects
        116574 Hot Object Cache Items
        33587476 on-disk objects

Adrian Chadd

unread,
Feb 4, 2009, 5:11:36 PM2/4/09
to lusca...@googlegroups.com
2009/2/4 Faysal Banna <degr...@gmail.com>:
> This is another stat ....
> behold that its capable of something of around 140 req/sec
> but when request goes over this limit i start to see degradation in
> performance ....

Are you graphing general system performance? Disk IO? memory/swap
utilisation? etc?

> by the way what are those errors i see in my cache.log as performance starts
> to degrade drastically ....?

Gah, just turn off store update:

update_headers off

The store update code is completely horrible and needs to be torn out.

Adrian Chadd

unread,
Feb 4, 2009, 5:12:05 PM2/4/09
to lusca...@googlegroups.com
Lusca-1.0 is cacheboy-1.6 renamed. :) You can just update to Lusca-1.0
whenever you'd like to.



Adrian

2009/2/4 Asif Bakali <asifb...@gmail.com>:

Faysal Banna

unread,
Feb 5, 2009, 12:32:39 AM2/5/09
to lusca...@googlegroups.com
Thanks for the tip Adrian
update_headers off
can you please put some notes on favorable parameters to use and what to avoid ??
by the way it was not used in my squid.conf so it was enabled by default inside the cache engine.... and i wonder why since it has a terrible impact ????

Are you graphing general system performance? Disk IO? memory/swap
utilisation? etc?
no i have no graphing tool on the system yet ... which do u recommend ???

Regards

Adrian Chadd

unread,
Feb 5, 2009, 1:11:58 AM2/5/09
to lusca...@googlegroups.com
"munin". Good place to start.

2009/2/5 Faysal Banna <degr...@gmail.com>:

Faysal Banna

unread,
Feb 5, 2009, 1:19:46 AM2/5/09
to lusca...@googlegroups.com
Adrian .
by now you know my needs do u have any recommendations to use COSS ????
what default values you prefer ?????

Regards

Adrian Chadd

unread,
Feb 5, 2009, 1:22:34 PM2/5/09
to lusca...@googlegroups.com
The defaults should be good to begin with. I'd configure COSS to cache
up to 128kbyte objects, and configure AUFS to cache objects 128kbytes
and over in size.



Adrian

2009/2/5 Faysal Banna <degr...@gmail.com>:

Faysal Banna

unread,
Feb 17, 2009, 3:55:58 AM2/17/09
to lusca...@googlegroups.com
Adrian

I compiled Squid-3.1 with tproxy and everything went great
yet i wanted to compile lusca with tproxy4 as you mentioned so i enabled like such

./configure --enable-linux-tproxy4    .............

this went ok
but when i do make

i get these errors and compilation fails
why ?????
In file included from comm_ips_tproxy4.c:16:
/usr/include/linux/netfilter_ipv4.h:53: error: 'INT_MIN' undeclared here (not in a function)
/usr/include/linux/netfilter_ipv4.h:64: error: 'INT_MAX' undeclared here (not in a function)
comm_ips_tproxy4.c: In function 'comm_ips_restoreCapabilities':
comm_ips_tproxy4.c:105: warning: implicit declaration of function 'debug'
comm_ips_tproxy4.c:105: error: called object 'debug(50, 1)' is not a function
make[1]: *** [comm_ips_tproxy4.o] Error 1
make[1]: Leaving directory `/var/tmp/lusca-1.0-r13795/libiapp'
make: *** [all-recursive] Error 1


Take into consideration i am using Linux 2.6.28 kernel which has already tproxy4 in it from balbit

Regards

Faysal Banna

unread,
Feb 17, 2009, 3:59:11 AM2/17/09
to lusca...@googlegroups.com
My ./configure for Lusca as for any squid is always the same as such below

./configure  '--prefix=/usr' '--exec_prefix=/usr' '--bindir=/usr/sbin' '--sbindir=/usr/sbin' '--libexecdir=/usr/lib/squid' '--sysconfdir=/etc/squid' '--localstatedir=/var/spool/squid' '--datadir=/usr/share/squid' '--enable-async-io' '--with-pthreads' '--enable-storeio=ufs,aufs,coss,diskd,null' '--enable-linux-netfilter' '--enable-arp-acl' '--enable-epoll' '--enable-removal-policies=lru,heap' '--enable-snmp' '--enable-delay-pools' '--enable-htcp' '--enable-cache-digests' '--enable-underscores' '--enable-referer-log' '--enable-useragent-log' '--enable-auth=basic,digest,ntlm,negotiate' '--enable-negotiate-auth-helpers=squid_kerb_auth' '--enable-carp' '--enable-follow-x-forwarded-for' '--with-large-files' '--with-maxfd=65536' 'amd64-debian-linux' '--enable-linux-tproxy4' 'build_alias=amd64-debian-linux' 'host_alias=amd64-debian-linux' 'target_alias=amd64-debian-linux' 'CFLAGS=-Wall -g -O2' 'LDFLAGS=-Wl,-Bsymbolic-functions' 'CPPFLAGS='


unless i am compiling for squid-3.1 which doesn't support coss nor null storage schemes and added only one option for Lusca as you pointed here Adrian "--enable-linus-tproxy4



Regards

Adrian Chadd

unread,
Feb 17, 2009, 11:21:03 AM2/17/09
to lusca...@googlegroups.com
I think I've seen a bug report submitted for this.

If someone would like to figure it out and submit a patch then please
be my guest. I'm still a day or two away from working on Lusca some
more (paid work needs completing..)



adrian

2009/2/17 Faysal Banna <degr...@gmail.com>:

Faysal Banna

unread,
Feb 19, 2009, 9:26:48 AM2/19/09
to lusca...@googlegroups.com
Ok i used the LUSCA_HEAD ... I have to admit 
compilation was successful yet.......
i didn't touch anything of the tproxy 5 rules found at balbit for iptables and iproute2
ip rule add fwmark 0x1/0x1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100
iptables -t mangle -N DIVERT
iptables -t mangle -A PREROUTING -p tcp -m socket -m physdev --physdev-in eth1 -j DIVERT

# DIVERT chain: mark packets and accept
iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT

iptables -t mangle -A PREROUTING -p tcp --dport 80 -m physdev --physdev-in eth1 -j
TPROXY --tproxy-mark 0x1/0x1 --on-port 33128

and as stated and was in squid.conf
http_port 33128 tproxy

but alas it was a failure
i switched back to squid-3.1 and here it was no changes and everything running fine ....

can someone test the configuration and say how it works for if you verify it works
then it has another method other than stated at balbit website -;


Regards

Alexandre Correa

unread,
Feb 19, 2009, 10:51:04 AM2/19/09
to lusca...@googlegroups.com
tproxy 2 or troxy4  ?!
--

Sds.
Alexandre J. Correa
Onda Internet / OPinguim.net
http://www.ondainternet.com.br
http://www.opinguim.net

Faysal Banna

unread,
Feb 19, 2009, 10:56:11 AM2/19/09
to lusca...@googlegroups.com
tproxy4

Faysal Banna

unread,
Feb 19, 2009, 1:12:53 PM2/19/09
to lusca...@googlegroups.com
why do i get high readings with these values
Median Service Times (seconds)  5 min    60 min:
        HTTP Requests (All):   1.71839  1.71839
        Cache Misses:          1.62803  1.62803
        Cache Hits:            2.50793  2.64709
        Near Hits:             4.54239  5.34113
        Not-Modified Replies:  0.85130  0.94847

        DNS Lookups:           0.00000  0.00000
        ICP Queries:           0.00000  0.00000

check the cache hits its very high and the near hits as well
regards
besides where does the DNS Lookups: statistics go ????

Adrian Chadd

unread,
Feb 19, 2009, 4:06:13 PM2/19/09
to lusca...@googlegroups.com
I don't know why the service times are high.

The DNS lookups are because I moved the DNS code out from src/ into a
seperate library so it can be tested/improved much easier; I've
probably just disabled the statistics counting.


2009/2/19 Faysal Banna <degr...@gmail.com>:

Alexandre Correa

unread,
Feb 19, 2009, 5:09:21 PM2/19/09
to lusca...@googlegroups.com
i´m testing dns statistics code .. and running fine with patches

see: http://code.google.com/p/lusca-cache/issues/detail?id=8

and i will commit this changes later ....

here my response times

Median Service Times (seconds)  5 min    60 min:
	HTTP Requests (All):   0.15888  0.15888
Cache Misses: 0.30459 0.30459
Cache Hits: 0.00000 0.00000
Near Hits: 9.67302 8.22659
Not-Modified Replies: 0.00000 0.00000
DNS Lookups: 0.03532 0.00372
ICP Queries: 0.00000 0.00000

Faysal Banna

unread,
Feb 20, 2009, 4:17:15 AM2/20/09
to lusca...@googlegroups.com
Wow look at Neah Hits

this is bad for performance as i have experienced it

Regards

Adrian Chadd

unread,
Feb 20, 2009, 10:06:55 AM2/20/09
to lusca...@googlegroups.com
2009/2/20 Faysal Banna <degr...@gmail.com>:

> Wow look at Neah Hits
>
> this is bad for performance as i have experienced it

Well, its either a local issue, or its with the codebase. I am not
sure whether its a Squid or Lusca problem. If you run up Squid-2.7 or
Squid-3.1, do the near miss times change?

Adrian

Faysal Banna

unread,
Feb 20, 2009, 11:35:30 AM2/20/09
to lusca...@googlegroups.com
No i have seen the same in all squid-2.7 squid-3.1 and lusca ... so actually i don't know why ....


Regards

Faysal Banna

unread,
Feb 20, 2009, 10:26:03 PM2/20/09
to lusca...@googlegroups.com
Adrian my friend
can u give a full command line on how do u configure coss  as in the configuration squid.conf

I use
cache_dir /cache1/coss 50000 max-size=1000000 maxfullbufs=4 membufs=20 block-size=4096
is this good or i should modify it bear in mind my system should hold something around 30Mbit/s with around 400 request/sec
and what is the next cache_dir aufs statement as you would put it ???
thanks in advance

Adrian Chadd

unread,
Feb 21, 2009, 1:06:31 AM2/21/09
to lusca...@googlegroups.com
Ok, then its most likely something weird with your local configuration.
Are you graphing stuff like disk IO rates? It sounds like it could be
overloaded disks.



Adrian

2009/2/20 Faysal Banna <degr...@gmail.com>:

Adrian Chadd

unread,
Feb 21, 2009, 1:08:13 AM2/21/09
to lusca...@googlegroups.com
I'm not using coss anywhere in production I'm afraid, I haven't in a few years.
I'll hopefully be doing so soonish, but not right now.

But a few notes:

* don't set max-size to 1000000, coss has no benefits for files that
big. Just set it to 65536 or 131072.
* how many physical disks would you use?

You should use the min-size= option in the aufs config, -after- the
coss config(s), to make sure that you don't fill AUFS partitions with
lots of small objects.



Adrian

2009/2/20 Faysal Banna <degr...@gmail.com>:

Faysal Banna

unread,
Feb 21, 2009, 6:26:03 AM2/21/09
to lusca...@googlegroups.com
ok .....
i shall use as you said :)
in your opinion should i see much difference in performance using coss ??? especially on my high load system
what am concerned here is the load of the requests/sec as i have never new anyone posting a req/s > 200
i have 4x512GB harddrives

Regards

Adrian Chadd

unread,
Feb 21, 2009, 3:32:35 PM2/21/09
to lusca...@googlegroups.com
AUFS should work fine on 4 512GB cache hard disks at 200 req a sec.
Thats only 50 req/sec per disk.

That said, if you use one COSS disk (yes, only one) to start with, and
set it up with say 80 gig or so of storage, you may notice a marked
improvement in performance.

2c,



Adrian

2009/2/21 Faysal Banna <degr...@gmail.com>:

Faysal Banna

unread,
Feb 21, 2009, 7:26:58 PM2/21/09
to lusca...@googlegroups.com
Great ...
with these info in mind i shall be posting you with another info about the performance i am getting
i need a c/c++ fragment on how to check the filesize and return it ....

i am doing a redirector to continue my previous work discussed earlier.... and i need a very fast file size checker throughh the web

like a program that would work as such
#!/bin/bash
FileSize=`myprog http://www.google.com/images/nav_logo4.png`

and thus $FileSize would now contain the size of nav_logo4.png which is equal to 7121 bytes

much regards
Reply all
Reply to author
Forward
0 new messages