how to see goog-malware-hash in readable format

[Mojo68116]

Currently we get both goog-malware-hash and goog-black-hash updates
and everything works like a champ. List is however coming back to us
as a key/value hash. Now we need this data in readable format.

Is there anyway I can lookup so that I can get this in clear text or
if there is a way to decode this ? Or like my suspicion may be its not
possible because its one way hash.

Any help would be appreciated ?

Thanks

Manoj

[Garrett Casto]

This is a one way hash. We don't want to expose the data for a few reasons, but one of them is that releasing a list of infected webservers is basically inviting bad guys to attack those machines, because they are known to be vulnerable.

Garrett

P.S. Your are going to want to change to using v2 of the protocol (http://code.google.com/apis/safebrowsing/developers_guide_v2.html) as v1 is going away in a few months.

--
You received this message because you are subscribed to the Google Groups "Google Safe Browsing API" group.
To post to this group, send email to google-safe-...@googlegroups.com.
To unsubscribe from this group, send email to google-safe-browsi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-safe-browsing-api?hl=en.

[Kiula]

On Apr 6, 10:11 am, Garrett Casto <gca...@google.com> wrote:
> This is a one way hash. We don't want to expose the data for a few reasons,
> but one of them is that releasing a list of infected webservers is basically
> inviting bad guys to attack those machines, because they are known to be
> vulnerable.

I have a couple of related questions:

(1) What's the difference between the BL and the Malware list? I tried
to search online but it's hard to find a simple page that explains the
difference.

(2) What's the best way to use this list on high traffic websites
where individual queries of URLs are not tenable. I have downloaded
the hash database too, for both Mal and BL, and set it up with the new
@gsb rule with the latest Mod_Security 2.6.0. However, this takes up a
lot of memory as the entire malware file of 12.5MB is loaded into
memory. It has crashed our Apache a few times. I'm wondering if we can
only use the 3MB BL file and be ok?

Welcome thoughts!

[Garrett Casto]

On Sun, May 22, 2011 at 8:27 AM, Kiula <phoeni...@gmail.com> wrote:

On Apr 6, 10:11 am, Garrett Casto <gca...@google.com> wrote:
> This is a one way hash. We don't want to expose the data for a few reasons,
> but one of them is that releasing a list of infected webservers is basically
> inviting bad guys to attack those machines, because they are known to be
> vulnerable.

I have a couple of related questions:

(1) What's the difference between the BL and the Malware list? I tried
to search online but it's hard to find a simple page that explains the
difference.

I'm not sure what you mean by this question. We have a few different lists, the main ones for API users being "goog-malware-shavar" and "googpub-phish-shavar". The former contains malware pages and the latter phishing pages.  I (and possibly our documentation) will occasionally use the word "blacklist" to mean add to either of these lists.

  

(2) What's the best way to use this list on high traffic websites
where individual queries of URLs are not tenable. I have downloaded
the hash database too, for both Mal and BL, and set it up with the new
@gsb rule with the latest Mod_Security 2.6.0. However, this takes up a
lot of memory as the entire malware file of 12.5MB is loaded into
memory. It has crashed our Apache a few times. I'm wondering if we can
only use the 3MB BL file and be ok?

From the sizes your quoting I'm assuming by BL you mean phishing blacklist. Obviously you can do this, but you aren't protecting yourself against malware in that case. Another option might be to use a whitelist before using the lookup service (so you don't send guaranteed good sites to us like "google.com", etc.). For malware this is slightly complicated because even large websites will occasionally get hacked, but it's better than not checking any requests at all.  We also should probably set caching headers for Loookup API requests, which might help you some as well. I'll put that somewhere on my TODO list.

Hope that helps

Garrett

Welcome thoughts!

[Kiula]

> I'm not sure what you mean by this question. We have a few different lists,
> the main ones for API users being "goog-malware-shavar" and
> "googpub-phish-shavar".

Thanks Garrett. Two questions:

1. Does it work if I combine the two files into one? The formats
appear to be the same. I am interested in local checking of hashes
against the database.

2. Is the format md5? How can I generate a hash of my own from the
URLs and then check against the local database?

Thanks.