Safe Browsing API incoherent results

[AB]

When testing a malicious URL using Google Safe Browsing APIs (v4) (https://developers.google.com/safe-browsing/v4/) we don't get any positive result. 

However, the very same URL with other Google services (e.g. https://transparencyreport.google.com/safe-browsing/search or Gmail) is explicitly and correctly marked as unsafe.

See image below showing different results for the same URL.

Google Safe Browsing API results are not coherent with other Google services, hence they do not provide useful or updated insights on malicious URLs, at least in our case.

Any idea why this incoherence?

[Alex Wozniak]

Hi,

Thanks for your message.

Due to data sharing restrictions, the set of URLs accessible via the Safe Browsing API, Transparency Report, and web browser integrations may differ. It is our goal to ensure these discrepancies are as rare as possible, but it's not guaranteed.

Thanks,

Alex

--
You received this message because you are subscribed to the Google Groups "Google Safe Browsing API" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-safe-browsi...@googlegroups.com.
To post to this group, send email to google-safe-...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-safe-browsing-api.
For more options, visit https://groups.google.com/d/optout.

[Andrei]

Hello,

Sorry to revive such an old thread, but I've stumbled across this problem as well, and there's a lot of domains that appear listed on the Transparency Report web interface, but do not appear as listed at all via the v4 API call.

Can something be done about this issue? As it renders the API quite useless, considering it doesn't return a lot of the dangerous/malicious domains.

Thank you.

On Monday, February 26, 2018 at 6:31:35 PM UTC+2, Alex Wozniak wrote:

Hi,

Thanks for your message.

Due to data sharing restrictions, the set of URLs accessible via the Safe Browsing API, Transparency Report, and web browser integrations may differ. It is our goal to ensure these discrepancies are as rare as possible, but it's not guaranteed.

Thanks,

Alex

On Mon, Feb 26, 2018 at 12:51 AM AB <alb...@badiapp.com> wrote:

When testing a malicious URL using Google Safe Browsing APIs (v4) (https://developers.google.com/safe-browsing/v4/) we don't get any positive result. 

However, the very same URL with other Google services (e.g. https://transparencyreport.google.com/safe-browsing/search or Gmail) is explicitly and correctly marked as unsafe.

See image below showing different results for the same URL.

Google Safe Browsing API results are not coherent with other Google services, hence they do not provide useful or updated insights on malicious URLs, at least in our case.

Any idea why this incoherence?

--
You received this message because you are subscribed to the Google Groups "Google Safe Browsing API" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-safe-browsing-api+unsub...@googlegroups.com.

[Alex Wozniak]

Hi Andrei,

Thanks for the message, and sorry to hear about your frustration.

We don't expect the Safe Browsing API and Transparency Report to differ often, but we'd be more than happy to take a look at any examples you come across that do not meet your expectations.

Thanks,

Alex

On Sun, Jun 3, 2018 at 11:15 AM Andrei <endur...@gmail.com> wrote:

Hello,

Sorry to revive such an old thread, but I've stumbled across this problem as well, and there's a lot of domains that appear listed on the Transparency Report web interface, but do not appear as listed at all via the v4 API call.

Can something be done about this issue? As it renders the API quite useless, considering it doesn't return a lot of the dangerous/malicious domains.

Thank you.

On Monday, February 26, 2018 at 6:31:35 PM UTC+2, Alex Wozniak wrote:

Hi,

Thanks for your message.

Due to data sharing restrictions, the set of URLs accessible via the Safe Browsing API, Transparency Report, and web browser integrations may differ. It is our goal to ensure these discrepancies are as rare as possible, but it's not guaranteed.

Thanks,

Alex

On Mon, Feb 26, 2018 at 12:51 AM AB <alb...@badiapp.com> wrote:

When testing a malicious URL using Google Safe Browsing APIs (v4) (https://developers.google.com/safe-browsing/v4/) we don't get any positive result. 

However, the very same URL with other Google services (e.g. https://transparencyreport.google.com/safe-browsing/search or Gmail) is explicitly and correctly marked as unsafe.

See image below showing different results for the same URL.

Google Safe Browsing API results are not coherent with other Google services, hence they do not provide useful or updated insights on malicious URLs, at least in our case.

Any idea why this incoherence?

--
You received this message because you are subscribed to the Google Groups "Google Safe Browsing API" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-safe-browsi...@googlegroups.com.

To post to this group, send email to google-safe-...@googlegroups.com.

Visit this group at https://groups.google.com/group/google-safe-browsing-api.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Google Safe Browsing API" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-safe-browsi...@googlegroups.com.

[Andrei]

Hello,

I tried providing some examples, but my post got removed.

Please let me know how I can get those examples to you directly.

Thank you for your time.

To unsubscribe from this group and stop receiving emails from it, send an email to google-safe-browsing-api+unsub...@googlegroups.com.

To post to this group, send email to google-safe-...@googlegroups.com.

Visit this group at https://groups.google.com/group/google-safe-browsing-api.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Safe Browsing API" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-safe-browsing-api+unsub...@googlegroups.com.

[Alex Wozniak]

Hi Andrei,

If you obfuscate the examples such that they don't make a well-formed URL, and remove any hyperlinks, that should bypass the Google Groups message filter.

e.g.

a [dot] com /path/1/2/index.html

To unsubscribe from this group and stop receiving emails from it, send an email to google-safe-browsi...@googlegroups.com.

To post to this group, send email to google-safe-...@googlegroups.com.

Visit this group at https://groups.google.com/group/google-safe-browsing-api.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Safe Browsing API" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-safe-browsi...@googlegroups.com.

To post to this group, send email to google-safe-...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-safe-browsing-api.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Safe Browsing API" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-safe-browsi...@googlegroups.com.

[Andrei]

Hello Alex,

Here's the examples I was referring to:

u8akdd5e9r5upa (dot) stream

santder (dot) internet-banking (dot) cf

Thanks for the assistance.

To unsubscribe from this group and stop receiving emails from it, send an email to google-safe-browsing-api+unsub...@googlegroups.com.

To post to this group, send email to google-safe-...@googlegroups.com.

Visit this group at https://groups.google.com/group/google-safe-browsing-api.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Safe Browsing API" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-safe-browsing-api+unsub...@googlegroups.com.

To post to this group, send email to google-safe-...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-safe-browsing-api.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Google Safe Browsing API" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-safe-browsing-api+unsub...@googlegroups.com.

[Eriq VanBibber]

Andrei,

my own experience with this same question resulted in an answer that the Transparency report works differently.

if you enter a url or domain in the TR, it will do a bunch of forward looking analysis in order to "rank" the safety of the host/url.

the GSB api is more exacting in the sense that it only returns a binary GOOD/BAD type of result for a given url or hostname.

I've asked if the Google team could provide a ranking API for domains/urls and it was taken as a feature suggestion at this point.

The way to think of this is in reverse of normal.  if a URL/host fails a GSB check, then it should be expected to also fail the TR check as well.  

I hope that helps.

[Rogin Neil De Guzman]

I am also currently experiencing this issue with this url: computerquestions.on.nimp.org

[Raja Sekar]

Sorry I've revive this thread. But the issue of discrepancy in API data and browser data is still evident and is there any workaround or solution for this.

[Jjoc o]

Hi!,

same problem with a url = subtitleseeker dot com

Note: obviously I did not find the solution in the group ...

[Ben Sanders]

In the case of subtitleseeker, only specific paths are being blocked, not the whole domain. Safe Browsing always tries to block the most specific urls that will provide adequate coverage. The Safe Browsing transparency report (if that is what you were comparing against) looks up to see if any paths on that domain are being blocked, whereas the SB API is trying to answer the question, "should I block this page". Different intents/aggregation, so it is expected in this case that they say different things.

Not trying to downplay issues that some have had, just that it is easy to misinterpret what these two different tools are saying about a site.