monitoring traffic on a webserver
Kenneth Gonsalves
any recommendations for a good tool to monitor traffic on a server?
--
regards
Kenneth Gonsalves
Associate
NRC-FOSS
http://nrcfosshelpline.in/web/
Jaidev Sridhar
>
> hi,
> any recommendations for a good tool to monitor traffic on a server?
I like MRTG.
-Jaidev
--
You can rent this space for only $5 a week.
Kenneth Gonsalves
> > any recommendations for a good tool to monitor traffic on a server?
>
> I like MRTG.
how does it compare with nagios?
--
regards
KG
http://lawgon.livejournal.com
Balachandran Sivakumar
On Sat, Jan 24, 2009 at 10:46 AM, Kenneth Gonsalves
<law...@thenilgiris.com> wrote:
>
>>
>> I like MRTG.
>
> how does it compare with nagios?
>
MRTG is a simple tool that gives a graphical representation of
the bandwidth utilisation. You can have daily, weekly, monthly and
yearly graphs.
This link(taken from MRTG website's demo link) shows what an MRTG output is like
http://www.switch.ch/network/operation/statistics/geant2.html
This link might help for the comparison of nargios and MRTG.
http://forums.burst.net/archive/index.php/t-260.html
--
Thank you
Balachandran Sivakumar
Arise Awake and stop not till the goal is reached.
Mail: benig...@gmail.com
Blog: http://benignbala.wordpress.com/
Balaji Narayanan
Hash: SHA1
Kenneth Gonsalves said the following on Saturday 24 January 2009 10:46 AM:
> On Friday 23 Jan 2009 11:03:52 pm Jaidev Sridhar wrote:
>>> any recommendations for a good tool to monitor traffic on a server?
>> I like MRTG.
>
> how does it compare with nagios?
>
Nagios is a service availability monitoring tool. It can alert based on
a plethora of conditions.
When you said traffic monitoring, it is a bit unclear on what exactly
you want to do. Do you want to mine the access / error logs and get some
pretty graphs. If yes, Webalizer or awstats might be of help.
MRTG is a graphic tool to monitor and graph pretty much everything
available over SNMP. I used it long back to monitor a couple of routers.
I am not very sure about its features.
- -b-
- --
http://balajin.net/blog
http://www.flickr.com/photos/balajijegan/
Getting there is only half as far as getting there and back.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFJfTI3Iar0Rc5ZY40RArsrAKCQCCEk5AUGBUTly2KdN/rpefstzACcDo8h
Hh8T3NyoVDsuXB8gh7g9fP0=
=rc82
-----END PGP SIGNATURE-----
Kenneth Gonsalves
> When you said traffic monitoring, it is a bit unclear on what exactly
> you want to do.
I am getting about 5-10 mails a day from the hosting company saying that
traffic is over the limit - want to know what all this traffic is, and where
is it coming from.
Vivek Khurana
>
> On Monday 26 Jan 2009 9:17:03 am Balaji Narayanan wrote:
>> When you said traffic monitoring, it is a bit unclear on what exactly
>> you want to do.
>
> I am getting about 5-10 mails a day from the hosting company saying that
> traffic is over the limit - want to know what all this traffic is, and where
> is it coming from.
In this case the hosting company should be able to provide you with
the logs. Also, check for the presence of bot on the server...
regards
Vivek
--
The hidden harmony is better than the obvious!!
Kenneth Gonsalves
> > I am getting about 5-10 mails a day from the hosting company saying that
> > traffic is over the limit - want to know what all this traffic is, and
> > where is it coming from.
>
> In this case the hosting company should be able to provide you with
> the logs. Also, check for the presence of bot on the server...
dedicated server - I have the logs. I want a tool to examine the logs. Anyway
am in the process of install nagios - hope it will solve my problems. btw,
how to detect a bot?
Vivek Khurana
>
> On Tuesday 27 Jan 2009 11:34:07 am Vivek Khurana wrote:
>> > I am getting about 5-10 mails a day from the hosting company saying that
>> > traffic is over the limit - want to know what all this traffic is, and
>> > where is it coming from.
>>
>> In this case the hosting company should be able to provide you with
>> the logs. Also, check for the presence of bot on the server...
>
> dedicated server - I have the logs. I want a tool to examine the logs. Anyway
> am in the process of install nagios - hope it will solve my problems. btw,
> how to detect a bot?
>
If you want to analyze offline logs then you should use awstats, it
will give you an idea of the traffic consumption based on protocol
(web, ftp, mail etc). Then drill down the graphs to see which sites
are connecting to you. For over traffic over limit you should be
looking at
1) Do you have any big files for download and are they downloaded
frequently. In that case check if any search engine crawl bot is
downloading the files on regular basis.
2) If you see lot of data going to other sites from your server then
someone might have installed a bot or a remote shell on your server.
Look at awstats site wise and volume wise logs... Nagios will not
provide you with much helpful information....
Sankarshan Mukhopadhyay
> any recommendations for a good tool to monitor traffic on a server?
Zabbix perhaps. <http://www.zabbix.com/licence.php> and,
<http://www.zabbix.com/features.php>
--
http://www.gutenberg.net - Fine literature digitally re-published
http://www.plos.org - Public Library of Science
http://www.creativecommons.org - Flexible copyright for creative work
Balaji Narayanan
Hash: SHA1
Kenneth Gonsalves said the following on Tuesday 27 January 2009 10:54 AM:
> On Monday 26 Jan 2009 9:17:03 am Balaji Narayanan wrote:
>> When you said traffic monitoring, it is a bit unclear on what exactly
>> you want to do.
>
> I am getting about 5-10 mails a day from the hosting company saying that
> traffic is over the limit - want to know what all this traffic is, and where
> is it coming from.
>
The first thing to start with is to analyse your access / error logs for
the webserver. Use webalizer or awstats to start with. That should give
you a clear picture on what is happening with regards to your webserver.
If the issue is not with webserver, it will need some drill down into
what else is causing the issue.
- -balaji
Getting there is only half as far as getting there and back.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFJfxU+Iar0Rc5ZY40RAkQzAJ9jfZkvbqOKL3sPktmALuYrdKJ/OACgqcAu
tXQKvrzqubLcooEOW+sQn3w=
=WK+w
-----END PGP SIGNATURE-----
Kenneth Gonsalves
> > I am getting about 5-10 mails a day from the hosting company saying that
> > traffic is over the limit - want to know what all this traffic is, and
> > where is it coming from.
>
> The first thing to start with is to analyse your access / error logs for
> the webserver. Use webalizer or awstats to start with. That should give
> you a clear picture on what is happening with regards to your webserver.
> If the issue is not with webserver, it will need some drill down into
> what else is causing the issue.
I installed awstats and find that it is not webserver traffic anyway - most of
the traffic on the site seems to be from google, yahoo, msn and other bots.
The rest is the expected traffic. I will have to start looking at mail -
awstats seems to have mail monitoring too.
Raj Shekhar
> hi,
> any recommendations for a good tool to monitor traffic on a server?
Saw the excellent suggestion from others about this. My own 2 cents on this
- ask for graphs from your hosting company for your traffic. Check if
you see any peaks in those graphs. Check at least a month's worth of
data to avoid any local maxima. Specifically ask for mrtg graphs - they
should be able to give it to you (mention that you suspect there might
be some bots on the box)
- filter traffic based on protocol (ftp/smtp/http) and see if you can
find which protocol is taking up your bandwidth. Your hosting company
might already have graphs
- enable sar or better still ask your hosting company to provide you
snmp data. examine your disk/cpu/network utilization and then see the
processes running at time when you see peaks in the resource utilization.
--
raj shekhar
facts: http://rajshekhar.net
opinions: http://rajshekhar.net/blog
I've never made anyone's life easier and you know it!
Kenneth Gonsalves
> - ask for graphs from your hosting company for your traffic.
dedicated server - am on my own ;-)
Vivek Khurana
<law...@thenilgiris.com> wrote:
>
> I installed awstats and find that it is not webserver traffic anyway - most of
> the traffic on the site seems to be from google, yahoo, msn and other bots.
> The rest is the expected traffic. I will have to start looking at mail -
> awstats seems to have mail monitoring too.
Hmm,,,drill down further and look what these bots are doing when they
crawl the site.. If you have files, check if these files are
downloaded by the bots. If a bot crawls and downloads all files during
every crawl, you will end up having wasting lot of bandwidth.
Ritesh Raj Sarraf
> Hmm,,,drill down further and look what these bots are doing when they
> crawl the site.. If you have files, check if these files are
> downloaded by the bots. If a bot crawls and downloads all files during
> every crawl, you will end up having wasting lot of bandwidth.
Assuming if that is the case, how would one go about fixing it ?
Ritesh
--
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."
Vivek Khurana
> On Thursday 29 Jan 2009 13:27:17 Vivek Khurana wrote:
>> Hmm,,,drill down further and look what these bots are doing when they
>> crawl the site.. If you have files, check if these files are
>> downloaded by the bots. If a bot crawls and downloads all files during
>> every crawl, you will end up having wasting lot of bandwidth.
>
> Assuming if that is the case, how would one go about fixing it ?
Modify your robots.txt :)