Using Abot2Demo:crawledPage.HttpRequestException. on https://aanhangwagenspattyn.be/ while ok in browser

36 views
Skip to first unread message

ghislain borremans

unread,
Feb 20, 2021, 5:48:35 AM2/20/21
to Abot Web Crawler
While in FireFox theis siteopens without problems, Abot is having a problem with it.
The message refers to an "invalid or unrecognized response"
Is this a setting problem or a bug?

thanks in advance

Ghislain

Did not crawl the links on page https://aanhangwagenspattyn.be/ due to Page has no content
ERR: Crawl of page failed crawledPage.HttpRequestException.InnerException = System.IO.IOException: The server returned an invalid or unrecognized response.
   at System.Net.Http.HttpConnection.FillAsync()
   at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
Page had no content https://aanhangwagenspattyn.be/

ghislain borremans

unread,
Feb 20, 2021, 6:16:50 AM2/20/21
to Abot Web Crawler
When using Postman,  the exact same request returns OK
The HttpClient.sendAsync in the PageRequester seems to cause the problem



Op zaterdag 20 februari 2021 om 11:48:35 UTC+1 schreef ghislain borremans:

sjdi...@gmail.com

unread,
Feb 22, 2021, 7:35:50 PM2/22/21
to ghislain borremans, Abot Web Crawler
Discussed in github issue

--
You received this message because you are subscribed to the Google Groups "Abot Web Crawler" group.
To unsubscribe from this group and stop receiving emails from it, send an email to abot-web-crawl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/abot-web-crawler/e1993d51-6543-4eef-940e-bb3b523b4d64n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages