Using Abot2Demo:crawledPage.HttpRequestException. on https://aanhangwagenspattyn.be/ while ok in browser

[ghislain borremans]

While in FireFox theis siteopens without problems, Abot is having a problem with it.

The message refers to an "invalid or unrecognized response"

Is this a setting problem or a bug?

thanks in advance

Ghislain

Did not crawl the links on page https://aanhangwagenspattyn.be/ due to Page has no content
ERR: Crawl of page failed crawledPage.HttpRequestException.InnerException = System.IO.IOException: The server returned an invalid or unrecognized response.
   at System.Net.Http.HttpConnection.FillAsync()
   at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
   at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
Page had no content https://aanhangwagenspattyn.be/

[ghislain borremans]

When using Postman,  the exact same request returns OK

The HttpClient.sendAsync in the PageRequester seems to cause the problem

Op zaterdag 20 februari 2021 om 11:48:35 UTC+1 schreef ghislain borremans:

[sjdi...@gmail.com]

Discussed in github issue

--
You received this message because you are subscribed to the Google Groups "Abot Web Crawler" group.
To unsubscribe from this group and stop receiving emails from it, send an email to abot-web-crawl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/abot-web-crawler/e1993d51-6543-4eef-940e-bb3b523b4d64n%40googlegroups.com.