For your reference, records indicate that
Eli the Bearded <*@
eli.users.panix.com> wrote:
> I don't
> have any reason to think Googlebot doing a GET on a .txt file is
> a malicious scan.
Since you aren’t in a business relationship with them for AdSense or
any other advertising service, there’s really no legitimate reason for
them to be scanning unpublished URLs like that. Save, of course, for
that fact they they’re looking to hoover up any and all information
about everyone they can get their hands on. I see them probing under
/.well-known/ and random 404 URLs as well. Google stopped playing
nice a long time ago.
> > It’s better to block their IP address completely. Even better, block
> > entire ranges by those “cloud” providers. Stop the abuse rather than
> > the notification of the problem.
>
> Advice like this I get can get from any hypochondriac webmaster forum.
> I'm perfectly capable of deciding what to block or not block on my own.
> My question was just about how ad agencies use ads.txt.
There’s plenty of information online about legitimate uses for that
file. What should concern you, since you don’t apparently buy or show
ads, is the improper uses. Same as for any other scans for invalid
URLs on your site. Serve up a blank file if you like. I personally
issue a 204 response for things like that, saving the bans for probes
that are directly going after exploit URLs.