Study Shows Mixed Performance Among Top Virus Scanning Programs
Simon Johnson
MELBOURNE, AUSTRALIA: Today Shake Communications Pty Ltd released the
results of its study of 20 of the top virus scanning programs.
The study tested the capacity of each product to detect various kinds of
viruses. Each program was pitted against a Virus Zoo of over 16,500
viruses, including Executable Files, Word Macro Viruses, Excel Macro
Viruses, Microsoft Access Viruses, Lotus 123 Viruses and also Trojans
and Bait Files.
Shake does not believe that virus scanning ability alone should be the
only, or most important, consideration in choosing an anti-virus
solution. Other features such as customer support, whether free upgrades
are available, and manageability, are also important. However, the study
does establish the raw capacity of each product to detect both "in the
wild" and "exotic" viruses.
Shake found that few virus scanning programs performed consistently
across all virus categories. In terms of the total number of viruses
detected, the top three products ranked as follows:
1. Anywhere AntiVirus
2. F-Secure
3. Norton AntiVirus
The entire report has been published in the September edition of the
Shake Security Journal. It is available by subscription at
http://www.shake.net
Background * Shake Communications Pty Ltd
Shake Communications is an independent IT Security company based in
Melbourne, Australia. Shake has the world's largest Vulnerabilities
Database containing vulnerabilities in hardware and software, including
patches and fixes. This enables companies to use their own staff to fix
vulnerabilities before hackers exploit them. A free demonstration can be
viewed online at http://www.shake.net/products/database/
The Shake Security Journal provides security professionals and IT
managers with cutting-edge, in-depth news, information and analyses on
recent events and developments in the security field. It is Australia's
only IT security journal, available by subscription. The March edition
can be viewed online for free at http://www.shake.net/products/journal/
ana...@hotmail.com
> Shake found that few virus scanning programs performed consistently
> across all virus categories. In terms of the total number of viruses
> detected, the top three products ranked as follows:
>
> 1. Anywhere AntiVirus
> 2. F-Secure
> 3. Norton AntiVirus
>
Who said that those products are "Top Virus Scanning Programs" ?
come on...
-----== Posted via Deja News, The Leader in Internet Discussion ==-----
http://www.dejanews.com/rg_mkgrp.xp Create Your Own Free Member Forum
Robert Hull
<simon....@shake.net> wrote
>Study Shows Mixed Performance Among Top Virus Scanning Programs
>
>MELBOURNE, AUSTRALIA: Today Shake Communications Pty Ltd released the
>results of its study of 20 of the top virus scanning programs.
>
[snip]
>
>Shake does not believe that virus scanning ability alone should be the
>only, or most important, consideration in choosing an anti-virus
>solution. Other features such as customer support, whether free upgrades
>are available, and manageability, are also important.
(not) they have found an *objective* method of comparing the help given
on a phone line.
[snip]
>the top three products ranked as follows:
>
>1. Anywhere AntiVirus
>2. F-Secure
>3. Norton AntiVirus
>
of AV suites. The results were as follows:
1 My Fay
2 Vrit Bittasoft
3 Wears Best
If you send me loadsa money, I will send you a publication in which (if
you search hard enough) you *might* find something about my evaluation
methods :)
--
Robert
F-Prot is available from http://www.complex.is
Dr Solomon's is still at http://www.drsolomon.com
AVP in Europe is at http://www.avp.ch
Angel V.
>Study Shows Mixed Performance Among Top Virus Scanning Programs
>Shake does not believe that virus scanning ability alone should be the
>only, or most important, consideration in choosing an anti-virus
>solution. Other features such as customer support, whether free upgrades
>are available, and manageability, are also important. However, the study
>does establish the raw capacity of each product to detect both "in the
>wild" and "exotic" viruses.
Man, you guys are in bad need of a clue! Let me put it this way. An
antivirus
company could have the very best customer support in the universe, they can
even offer free upgrades for eternity, and even bring me some lemonade when
I'm thirsty, but if their product's virus scanning ability has a lot to be
desired for,
you cannot possibly consider the product to be better than one that catches
everything or almost everything you throw at it, even if their support is
considered
lackluster. We are talking about ANTIVIRUS programs, not Corel Draw 8.
Virus scanning ability, detection, and disinfecting IS the most important
criteria!!
Alex B
AND the best Support together. If i had to choose between one of them,
logicaly i choose for the best detection.
--
Alex B - Mailto:ale...@bigfoot.com
I'm a happy AV-user !!!
--------------------------------------------------------
Angel V. wrote in message <6t8kld$dev$1...@ash.prod.itd.earthlink.net>...
>Simon Johnson wrote in message <35F5DAD5...@shake.net>...
>>Study Shows Mixed Performance Among Top Virus Scanning Programs
>Man, you guys are in bad need of a clue! Let me put it this way. An
Marcel D.
Alex B heeft geschreven in bericht <6t8ls9$rag$1...@news.worldonline.nl>...
Frederic
> [I'm sure you have already read it]
I miss Dr. Solomon's in the list. I would be very surprised
if it had suddenly stooped to an even lower level than
Norton.
Angel V.
>Personaly i think the best AV company is the one having the best detection
>AND the best Support together. If i had to choose between one of them,
>logicaly i choose for the best detection.
>
>--
>
>Alex B - Mailto:ale...@bigfoot.com
>
>I'm a happy AV-user !!!
>--------------------------------------------------------
Yes, I agree on that. The criteria that this particular publisher uses
though
is questionable. Their statement is like saying that a parachutist's most
important thing is not the parachute, but also his form
as he's plummeting towards earth, and the way he lands. But while these
may be true, they mean nothing if that parachute is not strapped to his
back.
His form at that point would be a hysterical scream, wild flapping of arms,
and a sickening splat on the ground. Not pretty. Yep...the parachute would
be
the most important thing there.
Jan Hruska
>results of its study of 20 of the top virus scanning programs.
>The study tested the capacity of each product to detect various kinds of
>viruses. Each program was pitted against a Virus Zoo of over 16,500
>viruses, including Executable Files, Word Macro Viruses, Excel Macro
>Viruses, Microsoft Access Viruses, Lotus 123 Viruses and also Trojans
>and Bait Files.
Could you specify where did you get your 'viruses' from? Did you verify
that all were viruses by replicating them yourself? How did you separate
viruses from the garbage which is present in even the best maintained
collections?
One of the common errors that non-specialists make when trying to separate
viruses from non-viruses is to run one or more virus scanners on the received
collection. If one (two/three etc) scanners say that a file contains a virus,
it is included in the test set. Of course, this is easier than replicating
every file, but as a methodology, it is flawed. This is especially true if
the scanners used for testing are run in high heuristics mode.
>Shake found that few virus scanning programs performed consistently
>across all virus categories. In terms of the total number of viruses
>detected, the top three products ranked as follows:
>1. Anywhere AntiVirus
>2. F-Secure
>3. Norton AntiVirus
Oh dear! Say, you don't have any connections with the UK's BrownWright, do
you? They are also an 'independent company' who published a study with a
similar flavour to it in April 1998.
Technical Director
Sophos Plc
rod
35F5DAD5...@shake.net...
>>Study Shows Mixed Performance Among Top Virus Scanning Programs
>
>MELBOURNE, AUSTRALIA: Today Shake Communications Pty Ltd
>released the results of its study of 20 of the top virus scanning programs.
>
>The study tested the capacity of each product to detect various kinds of
>viruses. Each program was pitted against a Virus Zoo of over 16,500
>viruses, including Executable Files, Word Macro Viruses, Excel Macro
>Viruses, Microsoft Access Viruses, Lotus 123 Viruses and also Trojans
>and Bait Files.
Without Shake's virus list, details of how the 16,500+ viruses in their
"Virus Zoo" were verified as _real_ and _infectious_ viruses, their list of
tested programs, and their testing methodology, this test has ZERO
credibility.
>Shake does not believe that virus scanning ability alone should be
>the only, or most important, consideration in choosing an anti-virus
>solution. Other features such as customer support, whether free
upgrades are available, and manageability, are also important.
Like many other antivirus "testers", Shake needs a reality injection.
It doesn't matter how good customer support is, or whether upgrades are free
or cost $500 a week, or how much "manageability" an antivirus program has
... the name of the game is "Find the Virus".
Bells and whistles and upgrades and fancy GUIs and instant customer
support aren't worth a damn if the program lets you down at detection time.
Any intelligent IT security man or woman would choose a plain vanilla DOS
command line scanner with an average detection rate of 95% of all viruses for
the past year over an all-singing all-dancing scan-your-email-as-you-type-it
scanner with an average detection rate of only 90% on the same viruses.
>However, the study does establish the raw capacity of each product to
>detect both "in the wild" and "exotic" viruses.
"In the wild" is NOT an indication of a scanner's _real world_ detection.
"Exotic"" is NOT an indication of a scanner's _real world_ detection.
OVERALL is the only worthwhile detection parameter.
>Shake found that few virus scanning programs performed consistently
>across all virus categories. In terms of the total number of viruses
>detected, the top three products ranked as follows:
>1. Anywhere AntiVirus
>2. F-Secure
>3. Norton AntiVirus
I find this very hard to believe unless the 16,500+ viruses used in the
test (assuming they _were_ all viruses) were provided by "Anywhere"
AntiVirus. (I presume the poster is referring to "Anyware" AntiVirus ...
I can't find "Anywhere" AntiVirus anywhere.)
MY quick and dirty tests of Anyware, F-Prot, F-Secure, Dr Solomon,
NAV, and NAI (on MANY more _verified_ viruses than 16,500) show
1-2-3 as:
1. AVP
1. F-Secure (using the AVP engine) (equal first)
2. Dr Solomon
3. NAI
Anyware was so far behind the others that one could safely assume that
several other available brands would have also beaten it in detection.
Granted, the version of Anyware I used was last updated on 21 July ... but
this is the latest version available on Anyware's website, which states "In
order to attain the new login and password, please send your serial/order
number (obtained with purchase of the Complete Version of Anyware
Antivirus) to upd...@helpvirus.com. We will then send you your personal
login and password. Trialware users will not have a serial / order number
as this only comes with the purchase of the Complete Version."
Assuming the licensed version is more up-to-date, one would expect it to
detect more viruses ... BUT ... the 21 July "freebie" doesn't detect Marburg
and CIH ... two viruses which were VERY widespread on 21 July.
Because I'm in the antivirus business my tests can't be regarded as 100%
impartial ... but I'm sure some of the acv virus buffs who aren't AVers
(and some VXers) also have more than 16,500 viruses at their disposal,
and I invite them to verify my findings.
Anyware AntiVirus is kinda hard to find ... even more so if you're looking
for "Anywhere" AntiVirus ... but I'll make life easier for anyone who wants
to verify my test results. You can download the latest demo from
ftp.helpvirus.com/pub/trialware/aew95.exe.
>The entire report has been published in the September edition of the
>Shake Security Journal.
I've been in the antivirus business for over ten years, and until today I'd
never heard of either Simon Johnson or the Shake Security Journal.
rod fewster
AVP Australia
www.avp.tm
rod
>35F5DAD5...@shake.net...
>
>>>Study Shows Mixed Performance Among Top Virus Scanning Programs
>>
>>MELBOURNE, AUSTRALIA: Today Shake Communications Pty Ltd
>>released the results of its study of 20 of the top virus scanning programs.
>>
>>The study tested the capacity of each product to detect various kinds of
>>viruses. Each program was pitted against a Virus Zoo of over 16,500
>>viruses, including Executable Files, Word Macro Viruses, Excel Macro
>>Viruses, Microsoft Access Viruses, Lotus 123 Viruses and also Trojans
>>and Bait Files.
Where did Shake get the Lotus 123 macroviruses ?
stev...@hotmail.com
"rod" <r...@st.net.au> wrote:
> Simon Johnson <simon....@shake.net> wrote in message
> 35F5DAD5...@shake.net...
>
> >>Study Shows Mixed Performance Among Top Virus Scanning Programs
> >
> >MELBOURNE, AUSTRALIA: Today Shake Communications Pty Ltd
> >released the results of its study of 20 of the top virus scanning programs.
> >
>
> >the only, or most important, consideration in choosing an anti-virus
> >solution. Other features such as customer support, whether free
> upgrades are available, and manageability, are also important.
>
> Like many other antivirus "testers", Shake needs a reality injection.
>
> It doesn't matter how good customer support is, or whether upgrades are free
> or cost $500 a week, or how much "manageability" an antivirus program has
> ... the name of the game is "Find the Virus".
>
> Bells and whistles and upgrades and fancy GUIs and instant customer
> support aren't worth a damn if the program lets you down at detection time.
>
> Any intelligent IT security man or woman would choose a plain vanilla DOS
> command line scanner with an average detection rate of 95% of all viruses for
> the past year over an all-singing all-dancing scan-your-email-as-you-type-it
> scanner with an average detection rate of only 90% on the same viruses.
>
I'm sorry, but I can't agree with this statement. In the average corporate
environment, a DOS command line scanner is about as useful as PERFECT.BAT -
half your staff would have no idea how to use it, and most of those that do
wouldn't bother. You can put in a good on-access scanner, and you'll still
get idiots who turn it off because it slows the computer down.
Virus detection IS the most important thing to consider, but it's not the
only thing. Stability of the software in your environment, usability,
manageability, scalability, cost of ownership and a whole host of other
factors have to be considered.
You want to make sure you never get a virus? Fine, just get rid of your PC.
If you want to minimize the impact that viruses have on your computing, then
you have to balance the steps you take to protect yourself against the impact
they have on your environment.
> >However, the study does establish the raw capacity of each product to
> >detect both "in the wild" and "exotic" viruses.
>
> "In the wild" is NOT an indication of a scanner's _real world_ detection.
>
> "Exotic"" is NOT an indication of a scanner's _real world_ detection.
>
> OVERALL is the only worthwhile detection parameter.
>
The wild list isn't perfect, but it's a start. Show me two scanners that
detect 95% of all viruses, where one of the detects 100% "in the wild", and
the other only 90%, and I know which one I'd consider to have the better
"real world" detection.
- Steve H, speaking only for myself
Frederic
> Man, you guys are in bad need of a clue! Let me put it this way. An
> antivirus
> company could have the very best customer support in the universe, they can
> even offer free upgrades for eternity, and even bring me some lemonade when
> I'm thirsty, but if their product's virus scanning ability has a lot to be
> desired for,
> you cannot possibly consider the product to be better than one that catches
> everything or almost everything you throw at it, even if their support is
> considered
> lackluster. We are talking about ANTIVIRUS programs, not Corel Draw 8.
> Virus scanning ability, detection, and disinfecting IS the most important
> criteria!!
Correct. When I'm sick, I swallow drugs, not some Beaujolais wine!
Frederic
> I'm sorry, but I can't agree with this statement. In the average corporate
> environment, a DOS command line scanner is about as useful as PERFECT.BAT -
> half your staff would have no idea how to use it, and most of those that do
> wouldn't bother. You can put in a good on-access scanner, and you'll still
> get idiots who turn it off because it slows the computer down.
Have you ever started Windows from a clean boot diskette?