Fwd: Internet Bandwidth Throttle using Squid Server
redhat linux
---------- Forwarded message ----------
From: amol kesare <amol_...@hotmail.com>
Date: Dec 7, 2005 10:54 PM
Subject: Internet Bandwidth Throttle using Squid Server
To: redha...@gmail.com
_________________________________________________________________
Tried the new MSN Messenger? It�s cool! Download now.
http://messenger.msn.com/Download/Default.aspx?mkt=en-in
*What is Bandwidth throttling :*
Consider we are having 512Kbps Internet connection line. And the system
administrator wants to divide the whole bandwidth into two sections. One for
the normal users and the other for special users who need high speed
internet connection. So that all the normal users can use 256 Kbps and the
special users can use the remaining 256 Kbps. So in this case you have to
make a bandwidth throttling to differentiate the whole single pipe line into
two sections. This can be achieved using squid proxy server.
Squid main configuration file ----- /etc/squid/squid.conf
Squid log file ----- /var/log/squid/access.log
Cache log file ----- /var/log/squid/cache.log
Bandwidth throttling in squid is done using Delay Pools. Delay pools uses
bucketing system.
*Throttle Calculations*
512Kbps(Kilo bits per second) -----> 64Kbytes -----> 64000bytes
256Kbps -----> 32Kbytes -----> 32000bytes
128Kbps -----> 16Kbytes -----> 16000bytes
64Kbps -----> 8Kbytes -----> 8000bytes
*Points to remember in Delay Pools:*
There are basically three things to note, they are - delay pools, delay
class, delay parameters.
Delay pool --- Defines how many pools we want to use
Delay Class ---- Defines type of the pool you are going to use.
Delay Parameter � allots the restrictions and fill rate/maximum bucket
size.
As I have said previously in this documentation delay pools uses bucketing
system.
Now there are three types of buckets
Class 1 pool: A single aggregate bucket, shared by all users
Class 2 pool: One aggregate bucket, 256 individual buckets
Class 3 pool: One aggregate bucket, 256 network buckets, 65,536 individual
buckets
If you still have any problem in the above syntax (ie) aggregate, network
and individual buckets to understand, then here is a simple syntax/example
for all these
*For Class 1 delay pool*
delay_parameters 1 32000/32000
*For Class 2 delay pool*
delay_parameters 1 48000/48000 48000/48000
*For Class 3 delay pool*
delay_parameters 2 32000/32000 8000/8000 16384/16384
*Example Setup:*
One 512Kbps pipe line. We want to distribute the whole pipe line into 2. One
for normal users and other for special users as follows
Normal users --- 128 Kbps
Special Users --- 384 Kbps
*Configuration Setup File /etc/squid/squid.conf:*
Before getting into the delay pools setup first create the acl(Access
Control List). Through acl you can define rules according to your
requirements. Find the Access Control section in the squid.conf file.
Here is an example:
acl superusers src 192.168.1.1 <http://192.168.1.1>
192.168.1.2< http://192.168.1.2>
192.168.1.3 <http://192.168.1.3>
acl mynetwork src
192.168.1.0/255.255.255.0<http://192.168.1.0/255.255.255.0>
Here in the above example only three users are special users who need
256Kbps bandwidth which are listed in the specialusers label and the whole
network including he special users are labeled as mynetwork.
After creating the users you have to allow them to access the internet.
Below line specifies for allowing the mentioned labeled users.
http_access allow superusers
http_access allow mynetwork
Now comes the funny part Delay Pools. Here We will deal with a basic example
for delay pools.
The example is as follows: We are having 512Kbps pipe line connection. We
want to divide it into segments one for the special users, web servers and
the other for the normal users in the organization. The special users and
the web servers are given 384Kbps speed and the remaining 128Kbps for the
normal users.
*Delay Pools for super users:*
Check for the delay pools section in the squid.conf file.
Start the configuration for delay pools as follows
==================================
########## Delay Pools############
==================================
delay_pools 2
As described above we have to create 2 delay pools, one for each delay
class.
====================================================
######### Defining Delay pool 1 in class 2 #########
====================================================
delay_class 1 2
delay_parameters 1 48000/48000 48000/48000
delay_access 1 allow superusers
The first line specifies Delay Class which defines delay pool 1 for the
delay class 2.
*Why we are using delay class 2 here?*
The first part on the second line ie �1� defines the pool One(1)
The second part on second line defines the aggregate
48000/48000(restore/max). where restore is the number of bytes (not bits -
modem and network speeds are usually quoted in bits) per second placed into
the bucket, and maximum is the maximum number of bytes which can be in the
bucket at any time.
The third part on second line is individual buckets again
48000/48000(restore/max). All the special users and the web servers should
obtain the same speed of 384 Kbps.
The third line allows the super users to fall in that bucket.
=====================================================
######### Defining Delay pool 2 in class 3 #########
=====================================================
*Delay Pools for normal users:*
delay_class 2 3
delay_parameters 2 32000/32000 8000/8000 16384/16384
delay_access 2 allow !superusers
The first line specifies Delay Class which defines delay pool 2 for the
delay class 3.
The first part on the second line ie �2� defines the pool Two(2) of
class
three(3)
The second part on second line defines the aggregate
32000/32000(restore/max) as aggregate (ie for whole).
*Note : - If you use -1/-1. �-1� indicates �unlimited�.*
The third part on second line is network bucket. We have defined here
8000/8000 .
The fourth part on second line is for individual.
The third line specifies to throttle all the users except the super users.
*How to check:*
*First set proxy settings in your browser as follows:*
Go to any client machine and open an IE(Internet Explorer). Select "Tools"
menu then select "Internet Options" then select "Connections" tab. After
that select the "LAN Settings" button, you will get a new pop-up window
there you select "Use proxy server for your LAN" check box and finally give
your squid proxy servers ip address and the port on which the squid proxy
server is configured. Now press "OK" and again press "OK". Cloase the window
and now open a new window.
*Second Check by downloading any large file*
Start to download any lage file not less than 10MB size. Because when ever
you download any small file it used to download it quickly. When you start
downloading large file at first if you note some starting bytes of the file
gets downloaded quickly and later it gets slowed down.
Keep a note on the download pop-up box which shows the speed limits in "*
kbps*". When we start downloading note on the pop-up box the speed limits in
*kbps* shows high value and then slowly gets decreased. This means when you
start downloading the bucket gets filled and slowly it decreases and becomes
stable to the specified limit in Kbps according to the configuration.
Remember again as i told restore value is used to set the download speed,
and the max value lets you set the size at which the files are to be slowed
down from.
--
Thanks and Warm Regards,
Brijesh Chougule
+91 9324756562
--
This is linux group mail box.
Gopal Gosavi
amol kesare
Hello.......
following are some imp dns setting which we r using....hope this is hope for
u all
. DNS Configuration Types
Most DNS servers are schizophrenic - they may be masters (authoritative) for
some zones, slaves for others and provide caching or forwarding for all
others. Many observers object to the concept of DNS types partly because of
the schizophrenic behaviour of most DNS servers and partly to avoid
confusion with the name.conf zone parameter 'type' which only allows master,
slave, stub, forward, hint). Nevertheless, the following terms are commonly
used to describe the primary function or requirement of DNS servers.
Contents
4.1 Master (a.k.a. Primary) DNS Server
4.2 Slave (Secondary) DNS Server
4.3 Caching (a.k.a. hint) DNS Server
4.4 Forwarding (a.k.a Proxy, Client, Remote) DNS Server
4.5 Stealth (a.k.a. DMZ or Split) DNS Server
4.6 Authoritative Only DNS Server
4.1 Master (Primary) Name Servers
A Master DNS contains one or more zone files for which this DNS is
Authoritative ('type master'). The zone has been delegated (via an NS
Resource Record) to this DNS.
The term 'master' was introduced in BIND 8.x and replaced the term
'primary'.
Master status is defined in BIND by including 'type master' in the zone
declaration section of the named.conf file) as shown by the following
fragment.
// example.com fragment from named.conf
// defines this server as a zone master
zone "example.com" in{
type master;
file "pri.example.com";
};
Notes:
1.
The terms Primary and Secondary DNS entries in Windows TCP/IP network
properties mean nothing, they may reflect the 'master' and 'slave'
name-server or they may not - you decide this based on operational need, not
BIND configuration.
2.
It is important to understand that a zone 'master' is a server which
gets its zone data from a local source as opposed to a 'slave' which gets
its zone data from an external (networked) source (the 'master'). This
apparently trivial point means that you can have any number of 'master'
servers for any zone if it makes operational sense. You have to ensure (by a
manual or other process) that the zone files are synchronised but apart from
this there is nothing to prevent it.
3.
Just to confuse things still further you may run across the term
'Primary Master' this has a special meaning in the context of dynamic DNS
updates and is defined to be the name server that appears in the SOA RR
record.
When a master DNS receives Queries for a zone for which it is authoritative
then it will respond as 'Authoritative' (AA bit is set in a query response).
When a DNS server receives a query for a zone which it is neither a Master
nor a Slave then it will act as configured (in BIND this behaviour is
defined in the named.conf file):
1. If caching behaviour is permitted and recursive queries are allowed the
server will completely answer the request or return an error.
2. If caching behaviour is permitted and Iterative (non-recursive) queries
are allowed the server can respond with the complete answer (if it is
already in the cache because of another request), a referral or return an
error.
3. If caching behaviour NOT permitted (an 'Authoritative Only' DNS server)
the server will return a referral or return an error.
A master DNS server can export (NOTIFY) zone changes to defined (typically
slave) servers. This ensures zone changes are rapidly propagated to the
slaves (interrupt driven) rather than rely on the slave server polling for
changes. The BIND default is to notify the servers defined in NS records for
the zone.
If you are running Stealth Servers and wish them to be notified you will
have to add an also-notify parameter as shown in the BIND named.conf file
fragment below:
// example.com fragment from named.conf
// defines this server as a zone master
// 192.168.0.2 is a stealth server NOT listed in a NS record
zone "example.com" in{
type master;
also-notify {192.168.0.2;};
file "pri/pri.example.com";
};
You can turn off all NOTIFY operations by specifying 'notify no' in the zone
declaration.
Example configuration files for a master DNS are provided.
4.2 Slave (Secondary) Name Servers
A Slave DNS gets its zone file information from a zone master and it will
respond as authoritative for those zones for which it is defined to be a
'slave' and for which it has a currently valid zone configuration.
The term 'slave' was introduced in BIND 8.x and replaced the term
'secondary'.
Slave status is defined in BIND by including 'type slave' in the zone
declaration section of the named.conf file) as shown by the following
fragment.
// example.com fragment from named.conf
// defines this server as a zone slave
zone "example.com" in{
type slave;
file "sec/sec.example.com";
masters {192.168.23.17;};
};
Notes:
1. The master DNS for each zone is defined in the 'masters' zone section
and allows slaves to refresh their zone record when the 'expiry' parameter
of the SOA Record is reached. If a slave cannot reach the master DNS when
the 'expiry' time has been reached it will stop responding to requests for
the zone. It will NOT use time-expired data.
2. The file parameter is optional and allows the slave to write the
transferred zone to disc and hence if BIND is restarted before the 'expiry'
time the server will use the saved data. In large DNS systems this can save
a considerable amount of network traffic.
Assuming NOTIFY is allowed in the master DNS for the zone (the default
behaviour) then zone changes are propagated to all the slave servers defined
with NS Records in the master zone file. There can be any number of slave
DNS's for any given 'master' zone. The NOTIFY process is open to abuse.
BIND's default behaviour is to only allow NOTIFY from the 'master' DNS.
Other acceptable NOTIFY sources can be defined using the allow-notify
parameter in named.conf.
Example configuration files for a slave DNS are provided.
4.3 Caching Name Servers
A Caching Server obtains information from another server (a Zone Master) in
response to a host query and then saves (caches) the data locally. On a
second or subsequent request for the same data the Caching Server will
respond with its locally stored data (the cache) until the time-to-live
(TTL) value of the response expires at which time the server will refresh
the data from the zone master.
If the caching server obtains its data directly from a zone master it will
respond as 'authoritative', if the data is supplied from its cache the
response is 'non-authoritative'.
The default BIND behaviour is to cache and this is associated with the
recursion parameter (the default is 'recursion yes'). There are many
configuration examples which show caching behaviour being defined using a
'type hint' statement in a zone declaration. These configurations confuse
two distinct but related functions. If a server is going to provide caching
services then it must provide recursive queries and recursive queries need
access to the root servers which is provided via the 'type hint' statement.
A caching server will typically have a named.conf file which includes the
following fragment:
// options section fragment of named.conf
// recursion yes is the default and may be omitted
options {
directory "/var/named";
version "not currently available";
recursion yes;
};
// zone section
....
// the DOT indicates the root domain = all domains
zone "." IN {
type hint;
file "root.servers";
};
Notes:
1. BIND defaults to recursive queries which by definition provides caching
behaviour. The named.conf recursion parameter controls this behaviour.
2. The zone '.' is shorthand for the root domain which translates to 'any
domain not defined as either a master or slave in this named.conf file'.
3. cache data is discarded when BIND is restarted.
The most common DNS server caching configurations are:
* A DNS server acting as master or slave for one or more zones (domains)
and as cache server for all other requests. A general purpose DNS server.
* A caching only local server - typically used to minimise external
access or to compensate for slow external links. This is sometimes called a
Proxy server though we prefer to associate the term with a Forwarding server
To cache or not is a crucial question in the world of DNS. BIND is regarded
as the reference implementation of the DNS specification. As such it
provides excellent - if complex to configure - functionality. The down side
of generality is suboptimal performance on any single function - in
particular caching involves a non-trivial performance overhead.
For general usage the breadth of BIND functionality typically offsets any
performance concerns. However if the DNS is being 'hit' thousands of times
per second performance is a major factor. There are now a number of
alternate Open Source DNS servers some of which stress performance. These
servers typically do NOT provide caching services (they are said to be
'Authoritative only' servers).
Example configuration files for a caching DNS are provided.
Note: The response to a query is Authoritative under three conditions:
1. The response is received from a Zone master.
2. The response is received from a Zone slave with non time-expired zone
data.
3. The response is received by a caching server directly from either a
Zone master or slave. If the response is read from the cache directly it is
not authoritative.
4.4 Forwarding (a.k.a Proxy) Name Servers
A forwarding (a.k.a. Proxy, Client, Remote) server is one which simply
forwards all requests to another DNS and caches the results. On its face
this look a pretty pointless exercise. However a forwarding DNS sever can
pay-off in two ways where access to an external network is slow or
expensive:
1. Local DNS server caching - reduces external access and both speeds up
responses and removes unnecessary traffic.
2. Remote DNS server provides recursive query support - reduction in
traffic across the link - results in a single query across the network.
Forwarding servers also can be used to ease the burden of local
administration by providing a single point at which changes to remote name
servers may be managed, rather than having to update all hosts.
Forwarding can also be used as part of a Split Server configuration for
perimeter defence.
BIND allows configuration of forwarding using the forward and forwarders
parameters either at a 'global' level (in an options section) or on a
per-zone basis in a zone section of the named.conf file. Both configurations
are shown in the examples below:
Global Forwarding - All Requests
// options section fragment of named.conf
// forwarders can have multiple choices
options {
directory "/var/named";
version "not currently available";
forwarders {10.0.0.1; 10.0.0.2;};
forward only;
};
// zone file sections
....
Per Domain Forwarding
// zone section fragment of named.conf
zone "example.com" IN {
type forward;
file "fwd.example.com";
forwarders {10.0.0.1; 10.0.0.2;};
};
Where dial-up links are used with DNS forwarding servers BIND's general
purpose nature and strict standards adherence may not make it an optimal
solution. A number of the Alternate DNS solutions specifically target
support for such links. BIND provides two parameters dialup and
heartbeat-interval (neither of which is currently supported by BIND 9) as
well as a number of others which can be used to minimise connection time.
Example configuration files for a forwarding DNS are provided.
4.5 Stealth (a.k.a. DMZ or Split) Name Server
A stealth server is defined as being a name server which does not appear in
any publicly visible NS Records for the domain. The stealth server is
normally used in a configuration called Split Severs which can be roughly
defined as having the following characteristics:
1. The organisation needs a public DNS to enable access to its public
services e.g. web, mail ftp etc..
2. The organisation does not want the world to see any of its internal
hosts either by interrogation (query or zone transfer) or should the DNS
service be compromised.
A Split Server configuration is shown in Figure 4.1.
Split (Stealth) Server configuration
Figure 4.1 Split Server configuration
The external server(s) is(are) configured to provide Authoritative Only
responses and no caching (no recursive queries accepted). The zone file for
this server would be unique and would contain ONLY those systems or services
that are publicly visible e.g. SOA, NS records for the public (not stealth)
name servers, MX record(s) for mail servers and www and ftp service A
records. Zone transfers can be allowed between between the public servers as
required but they MUST NOT transfer or accept transfers from the Stealth
server. While this may seem to create more work, the concern is that should
the host running the external service be compromised then inspection of the
named.conf or zone files must provide no more information than is already
publically visible. If 'master', 'allow-notify','allow-transfer' options are
present in named.conf (each of which will contain a private IP) then the
attacker has gained more knowledge about the organisation - they have
penetrated the 'veil of privacy'.
There are a number of articles which suggest that the view statement may be
used to provide similar functionality using a single server but this does
not address the problem of the DNS host system being compromised and by
simple inspection of the named.conf file additional data about the
organisation could be discovered. In our opinion 'view' does not provide
adequate security in a 'Split DNS' solution.
A minimal public zone file is shown below:
; public zone master file
; provides minimal public visibility of external services
example.com. IN SOA ns.example.com. root.example.com. (
2003080800 ; se = serial number
3h ; ref = refresh
15m ; ret = update retry
3w ; ex = expiry
3h ; min = minimum
)
IN NS ns1.example.com.
IN NS ns2.example.com.
IN MX 10 mail.example.com.
ns1 IN A 192.168.254.1
ns2 IN A 192.168.254.2
mail IN A 192.168.254.3
www IN A 192.168.254.4
ftp IN A 192.168.254.5
The internal server (the Stealth Server) can be configured to make visible
internal and external services, provide recursive queries and all manner of
other services. This server would use a private zone master file which could
look like this:
; private zone master file used by stealth server(s)
; provides public and private services and hosts
example.com. IN SOA ns.example.com. root.example.com. (
2003080800 ; se = serial number
3h ; ref = refresh
15m ; ret = update retry
3w ; ex = expiry
3h ; min = minimum
)
IN NS ns1.example.com.
IN NS ns2.example.com.
IN MX 10 mail.example.com.
; public hosts
ns1 IN A 192.168.254.1
ns2 IN A 192.168.254.2
mail IN A 192.168.254.3
www IN A 192.168.254.4
ftp IN A 192.168.254.5
; private hosts
joe IN A 192.168.254.6
bill IN A 192.168.254.7
fred IN A 192.168.254.8
....
accounting IN A 192.168.254.28
payroll IN A 192.168.254.29
Using BIND 9's view statement can provide different services to internal and
external requests can reduce further the Stealth server's visibility e.g.
forwarding all DNS internal requests to the external server.
Example configuration files for a stealth DNS are provided.
4.6 Authoritative Only Server
The term Authoritative Only is normally used to describe two concepts:
1. The server will deliver Authoritative Responses - it is a zone master
or slave for one or more domains.
2. The server will NOT cache.
There are two configurations in which Authoritative Only servers are
typically used:
1. As the public or external server in a Split (a.k.a. DMZ or Stealth) DNS
used to provide perimeter security.
2. High Performance DNS servers. In this context general purpose DNS
servers such as BIND may not provide an ideal solution and there are a
number of Open Source Alternatives some of which specialise in high
performance Authoritative only solutions.
You cannot completely turn off caching in BIND but you can control it and
provide the functionality described above by simply turning off recursion in
the 'option' section of named.conf as shown in the example below.
// options section fragment of named.conf
// recursion no = limits caching
options {
directory "/var/named";
version "not currently available";
recursion no;
};
// zone file sections
....
BIND provides three more parameters to control caching ,max-cache-size and
max-cache-ttl neither of which will have much effect on performance in this
particular case and allow-recursion which uses a list of hosts that are
permitted to use recursion (all others are not).
Example configuration files for a authoritative-only DNS are provided.
_________________________________________________________________
Shah Rukh fan? Know all about the Baadshah of Bollywood. On MSN Search
http://server1.msn.co.in/profile/shahrukh.asp
