Googlebot encountering 403 response codes

[Jay H.]

Googlebot is reporting nearly 3,000 403 response code errors on my site. I'm wondering the source. But first a little background…

• I'm using F3 v3.5.1-release
• When I "Fetch as Googlebot" everything works fine.
• I just recently let Googlebot into my site. So this is not something that was working and is now broken.
  
• I don't have F3's DNSBL feature enabled, so that's not the source of the 403s.
• I do use F3's session feature. 

I'm guessing it's a problem related to sessions. Another thread on here mentions that F3 senses session hijacking. How likely is it that that's the problem and how exactly does that work? 

I've disabled sessions when the user agent is googlebot. I'll see if that helps. But if anyone has other suggestions of what to look at, that would speed up my testing.

And, I know it's messy to put in exceptions to things, but googlebot is important enough that F3 shouldn't do anything to block it. (IMHO)

[ved]

First of all, I would recommend you try with the latest version from: https://github.com/bcosca/fatfree-core and see if that fixes the issue. 3.5.1 is pretty outdated in comparison.

By default F3 responds with http 403 if it detects a suspicious session (check: https://fatfreeframework.com/session#onsuspect) so I would recommend that you implement the onsuspect callback and use that to do some logging and/or googlebot detection to see if it helps you debug the issue. Alternatively you can also create your own F3 error handler and do the logging/debugging there.

[Jay H.]

Thanks. I'll try the onsuspect callback. That should help tell me what's going on. However, I'll stick with the release version. I'm conservative in that respect.