Hey,
Query on fair usage - I have 100,000 to 1m domains I need to query, and don't want to overload system. Do you have any guidelines on fair usage?
Would likely be looking at web scraping as I've found the database to be extremely slow (40s vs <1s for simple domain lookup).
I was thinking just perform requests sequentially, so not hammering in parallel and overloading, seems to be doing 1 every few seconds which seems to be inline with database suggestions of 2 connections per second - is that ok?
BTW great resource, thanks for your effort.
Best regards