Fortiguard Rating Lookup

[Boleslao Drinker]

You can configure DNS web filtering to allow, block, or monitor access to web content according to FortiGuard categories. When DNS web filtering is enabled, your FortiProxy unit must use the FortiGuard DNS service for DNS lookups. DNS lookup requests sent to the FortiGuard DNS service return with an IP address and a domain rating that includes the FortiGuard category of the web page.

If that FortiGuard category is set to block, the result of the DNS lookup is not returned to the requester. If the category is set to redirect, then the address returned to the requester points at a FortiGuard redirect page.

The DNS filter profile list can be viewed by selecting the List icon (the farthest right of the three icons in the upper right of the window; it resembles a page with some lines on it) in the Edit DNS Filter Profile page toolbar.

The DNS static domain filter allows you to block, exempt, or monitor DNS requests by using IPS to look inside DNS packets and match the domain being looked up with the domains on the static URL filter list. If there is a match the DNS request can be blocked, exempted, monitored, or allowed.

Hi We recently upgraded from 6.0.5 to 6.0.9 and then changed the Fortiguard protocol from UDP to HTTPS. Since then we are occasionally getting block messages 'An error occurred while trying to rate the website using the webfiltering service.

I've looked at the kb article =FD33528 but am slightly confused about the sentence "This will allow users to access the web sites when a rating error occurs, and will allow the FortiGate unit to use the FortiGuard Web Filtering database that it has stored on the unit to rate the web site." Can I just confirm that it means when it can't reach fortiguard it will rate the website using the local db and will allow/block access accordingly. Just looking at the option on the FG "Allow websites when a rating error occurs" suggests that it is going to allow it whatever the rating is. Thanks for your time

Keep in mind that the "local DB" cache is only a cache of the ratings for recently visited websites. It is a very small list compared to the full FortiGuard database. With this setting enabled there is a very good chance that traffic will get through to sites that would otherwise be blocked, so enable it at your own risk.

The reason that "a rating error occurs" happens more often with HTTPS vs. UDP is that Fortinet doesn't seem to have the same capacity to handle HTTPS web ratings lookups compared to UDP. If you run "diag debug rating" when in UDP mode vs. HTTPS mode you'll see that there are far more servers available to respond to UDP ratings lookups vs. HTTPS.

I can't say for sure, but I suspect this is a capacity issue with the FortiGuard rating servers getting overloaded occasionally and the HTTPS protocol is more sensitive to delays compared to a fast, connectionless protocol like UDP. Also, the default FortiGuard setting is to use anycast, and I've found that with anycast enabled, these errors seem to happen more often.

In my experience, if you're seeing a lot of "a rating error occurs" messages in the logs, use the UDP protocol and disable anycast for your FortiGuard settings. This is all covered by this Fortinet Tech Tip:

You're trading off security (encrypted HTTPS) vs. reliability (unencrypted UDP), but then again, if you have "allow websites when a rating error occurs" enabled, lack of reliability is a security issue. If you find that disabling anycast and/or enabling UDP resolves the ratings errors, ideally you would set "allow websites when a rating error occurs" back to disabled.

The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.

Most critical of them is Web Filter rating query - if your Fortigate cannot get answer what category the web site belongs to, access to this web site will be blocked by default. It means that if for any reason Fortigate cannot reach Fortiguard servers and it has security rules with Web Filtering by Category configured - those rules will BLOCK users access to ANY website, not just malicious ones.

First, check status of license/subscription and FortiGuard connection status in System -> FortiGuard - the Web Filtering status should be in green. This checks subscription license status, but not always detects connection to the FortiGuard status. If you see it red, it is most probably a license/subscription issue to be checked with Fortinet TAC, as subscription checks are done once in a while and are cached. To check actual connectivity to the FortiGuard servers - on the same page, under Filtering subsection, there is Test Connectivity button to push. It should return status as Up/green. Also pay attention to the widget on the same page in the right bottom corner FortiGuard Filter Rating Servers, it shows real time stats and IP addresses of the servers the Fortigate is trying to reach. If timings are unusually high and in red, there could be network connectivity problem, we will look at next.

Even better check is to run ping exe ping to all the hostnames above to see if the Fortigate can resolve AND can reach them. The most important of them being service.fortiguard.net.

Here:

Status - shows if Web Filtering as a service is enabled.

Protocol - via what protocol this Fortigate is trying to reach FortiGuard servers (more on this below).

Anycast - whether this Fortigate is trying to reach Anycast servers of FortiGuard (more on this below).

Server List - actual list of FortiGuard servers that this Fortigate was/is trying to reach. Here most important is status legend:

- F: failed, bad - Fortigate tried few times to reach this server to no avail. Note that it is bad only if ALL servers in the list have this status. It is OK if only few of the servers are unreachable.

- D: this server was successfully resolved from FQDN to its IP address, but it does not indicate its reachability yet.

- I: server to which Fortigate tries to initiate connection, most frequently goes with D,it does not indicate if a server is working or not yet.

- T: server was found, it answered, and is now being "timed", i.e. its answer time/RTT is being measured.

- TZ: Time Zone, while not a status indicator, Fortigate tries and prefers servers with the least time zone difference in hope of geographic proximity. Therefore, it is quite important to set correctly the time zone for your Fortigate.

Fortigate communicates for its functions with just one server at a time - the one on top of the list. The rest of the servers are being constantly monitored and their RTT, and packet loss measured. If the top-list server fails, it will be replaced with the next best one and so on. We do not have capability to influence this server list manually.

So if all servers in the list have F(ailed), what do we do next?. This may mean either all Fortiguard servers at the Fortinet side are down (less likely), or that this Fortigate has the problem of reaching them at the network level.

Fortigate can use several ports to talk to Fortiguard servers (or Fortiguard Distribution Network as they call it) - 53, 8888, 443, the default being 8888. The port 53 is a well known DNS protocol/port, only that Fortigate uses proprietary UDP/53 obfuscated/encrypted protocol to query the servers, and for this reason some IPS/anti-DDoS/etc protections on the way from Fortigate to FortiGuard may mark such traffic as malicious and drop it. You can check if it is the case by going to System -> FortiGuard -> Filtering and change (if set so) from port 53 to port 8888. On newer FortiOS versions (6.4 and up) they moved this to CLI only: config sys fortiguard then set port 538888443. So, as first debug measure it is recommended to try all possible ports and see if status of connection to the FortiGuard servers changes. Note about protocol I mentioned before - in 6.4 and newer they added option to force the communication to FortiGuard servers to be a valid HTTPS traffic, which is most likely to pass the Internet successfully. For this you have to enable it (in addition to setting port to 443) via CLI: config sys fortiguard, then set protocol https end.

Important note if you have VDOMs enabled - all communication to the Fortiguard network is initiated from management/root VDOM only! The frequent human error I've seen - someone by mistake changes management domain to the VDOM that has no/limited access to the Internet and as a consequence, it cannot reach FortiGuard network. Very common, indeed. To verify who is the management VDOM:

Anycast servers - starting with FortiOS 6.4 the default setting to reach FortiGuard is anycast. The intention was good - to improve reachability of FortiGuard servers, but unfortunately the implementation did not live up to the expectations. More often than not it actually creates a problem in reaching the Fortinet servers. It may improve in the future, but for now my advice is to disable anycast and switch back to unicast servers. You do so in CLI:

3a8082e126