Hi Hiskeyd,
You're more likely to catch the attention of sitemaps experts
(Googlers and non-Googlers) by posting this in the Sitemaps subgroup;
I welcome you to do that, since we're unfortunately unable to move
threads.
With that said... from my experience, I've seen situations like yours
when our bot tries to grab a robots.txt file and gets neither a 404
nor a legit robots.txt file, but rather a server error (e.g., 500, or
access forbidden). I'd check to see if there are other errors listed
within the Webmaster Console for your account and, if so, contact your
hosting provider to see if they're doing any throttling or blocking
which might result in these issues.
If you still experience this problem, I do invite you to repost your
note in the Sitemaps subgroup.