Using Abot2Demo:crawledPage.HttpRequestException. on while ok in browser

Skip to first unread message

ghislain borremans

Feb 20, 2021, 5:48:35 AM2/20/21
to Abot Web Crawler
While in FireFox theis siteopens without problems, Abot is having a problem with it.
The message refers to an "invalid or unrecognized response"
Is this a setting problem or a bug?

thanks in advance


Did not crawl the links on page due to Page has no content
ERR: Crawl of page failed crawledPage.HttpRequestException.InnerException = System.IO.IOException: The server returned an invalid or unrecognized response.
   at System.Net.Http.HttpConnection.FillAsync()
   at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
Page had no content

ghislain borremans

Feb 20, 2021, 6:16:50 AM2/20/21
to Abot Web Crawler
When using Postman,  the exact same request returns OK
The HttpClient.sendAsync in the PageRequester seems to cause the problem

Op zaterdag 20 februari 2021 om 11:48:35 UTC+1 schreef ghislain borremans:

Feb 22, 2021, 7:35:50 PM2/22/21
to ghislain borremans, Abot Web Crawler
Discussed in github issue

You received this message because you are subscribed to the Google Groups "Abot Web Crawler" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
To view this discussion on the web visit
Reply all
Reply to author
0 new messages