Http issues with github.com

[Robert Åkerblom-Andersson]

Hi,

We are having some issues connecting to Github from Dart, it started 2 days ago and we have not figured out where the problem is just yet. We have contacted Githubs support but they say that they can't see anything wrong on their end. We have debugged the issue so that we have a reproducible test that shows that something is wrong.

Try this code and you will see:

import 'dart:io';

import 'package:http/http.dart' as http;

void main() {

  // Gives expected output 

  var url1 = "https://www.google.com";

  var url2 = "https://www.github.com";

  

  // Test https://www.google.com

  http.get(url1).then((http.Response res) {

    print("Headers:");

    print(res.headers);

    print("");

    print("Body:");

    print(res.body);

    print("\n");

  })

  // Test https://www.github.com

  .then((_) => http.get(url2)).then((http.Response res) {

   print("Headers:");

   print(res.headers);

   print("");

   print("Body:");

   print(res.body);

  });

}

It works as expected for "https://www.google.com" but  for "https://www.github.com" the output says:

Uncaught Error: RedirectException: Redirect loop detected

Unhandled exception:

RedirectException: Redirect loop detected

#0      _rootHandleUncaughtError.<anonymous closure>.<anonymous closure> (dart:async/zone.dart:700)

#1      _asyncRunCallbackLoop (dart:async/schedule_microtask.dart:23)

#2      _asyncRunCallback (dart:async/schedule_microtask.dart:32)

#3      _asyncRunCallback (dart:async/schedule_microtask.dart:36)

#4      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:119)

The test is just a simple GET request to the two different domains and one works and one don't. In curl or a browser both urls work of course. We have tried both HttpClient and the http lib. 

It would be nice if someone else could test the script above to confirm that it does not work, so it's nothing special with out setup even though I doubt that. Any ideas of how to debug this further? 

Regards, Robert 

[Ivan Zaera Avellon]

Tested. (Next time, please send pubspec.yaml ;-)).

I also get the fail (from Madrid, Spain, if that matters).

But then I ran:

wget "https://www.github.com"

And I got:

Resolviendo www.github.com (www.github.com)... 192.30.252.131
Conectando con www.github.com (www.github.com)[192.30.252.131]:443... conectado.
ERROR: El certificado de “www.github.com” no es confiable.
ERROR: El certificado de “www.github.com” no tiene un emisor conocido.

Which means that the certificate for github is not valid because it has an unknown issuer. May be that?

El 21/03/14 13:02, Robert Åkerblom-Andersson escribió:

--
For other discussions, see https://groups.google.com/a/dartlang.org/
 
For HOWTO questions, visit http://stackoverflow.com/tags/dart
 
To file a bug report or feature request, go to http://www.dartbug.com/new
To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

[Ivan Zaera Avellon]

Strangely I'm all day long connected to github from the browser and the certificate is reported as valid there.

:-O

El 21/03/14 13:15, Ivan Zaera Avellon escribió:

[Robert Åkerblom-Andersson]

Hi Iván,

Thanks for testing it! Yeah, my bad, I should have included a pubspec.yaml.

Strange that you get certificate issues, I don't get that. However if it would have been a certificate issue I think there should have been a different error in dart. Plus, other github domains such as "help", "api" and "pages" work and they seems to have certificates from the same issuer.

Here is a different test program that uses HttpClient and tests 6 different urls (you have to manually switch the "urlX" parameter, and no pubspec.yaml needed):

import 'dart:io';

void main() {

  var url1 = "https://www.google.com";

  var url2 = "https://api.github.com";

  var url3 = "https://pages.github.com";

  var url4 = "https://help.github.com";

  var url5 = "https://github.com";

  var url6 = "https://www.github.com";

  

  // Make request 

  new HttpClient().getUrl(Uri.parse(url6)).then((HttpClientRequest request) {

    print(request.connectionInfo.remoteAddress);

    print(request.connectionInfo.remotePort);

    

    request.close().then((HttpClientResponse response) {

     List<int> resBuffer = new List<int>();

     var r = response.listen(null);

     

     print("Response headers:");

     response.headers.forEach((key, val) => print("  $key $val"));

     

     r.onData((data) => resBuffer.addAll(data));

     r.onDone(() {

       print("Response body:");

       print("  ${new String.fromCharCodes(resBuffer)}");

     });

     r.onError((e) => print(e));

    });

  });

}

Urls 1, 2, 3, 4 works just fine as expected. Url 5 and 6 does not work and only gives a redirection loop. This is the output of url5 to be more specific:

InternetAddress('192.30.252.129', IP_V6)

443

Uncaught Error: RedirectException: Redirect loop detected

Unhandled exception:

RedirectException: Redirect loop detected

#0      _rootHandleUncaughtError.<anonymous closure>.<anonymous closure> (dart:async/zone.dart:700)

#1      _asyncRunCallbackLoop (dart:async/schedule_microtask.dart:23)

#2      _asyncRunCallback (dart:async/schedule_microtask.dart:32)

#3      _asyncRunCallback (dart:async/schedule_microtask.dart:36)

#4      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:119)

Feels like there must be something on Github's side of things, but I'm not 100% sure on what and how to debug further. It just stopped working 2 days ago so it's really weird.

Regards, Robert

[Ivan Zaera Avellon]

People here at Liferay (were I work) are having issues with github too. And now it's failing for me.

I'm not sure, but it looks like someone is playing with DNS and/or certificates. If github is not doing anything on their side it may well be an attack.

Can you tell them about this? Just in case...

Regards,
Ivan

El 21/03/14 13:36, Robert Åkerblom-Andersson escribió:

[Robert Åkerblom-Andersson]

Hi Ivan and thanks for the update.

It seems like they have some attack according to the status page, so I think they know something is not right:

https://status.github.com/

However, the status page was okay yesterday and then the dart script did not work either.

Regards, Robert 

[William Hesse]

We have found the problem, and are working on a fix.  The problem is that the hostname in the request is being sent to the server as "www.githup.com:443" or "github.com:443", and it doesn't like the 443.  It redirects to the host without the 443, and we add it again when sending.  We need to drop the 443, like we drop the :80 for http requests.  If that is what the RFCs say.

--
William Hesse

[Robert Åkerblom-Andersson]

Hi William,

Good to hear that you found the problem. How soon do you think it could be fixed? 

I understand if it would take some time, but if it could be fixed today that would be great. We are trying to get DartVoid's beta started very soon and this is a blocking issue right now since users can't login with Github anymore...

If there is anything we can do to help, just say the words. I'm not sure what RFC to read to find the specifics in if the 443 should be dropped or not.

Regards, Robert

[Robert Åkerblom-Andersson]

Hi again,

I just got an update from Ivan Žužak (@Github) that I'we contacted about this and they seem to have found the issue!

Qoute from the mail:

I think we figured this out. It seems that HttpClient in Dart doesn't set the Host HTTP header be default. Recently, we started requiring that header on our side, which caused the difference in behavior you observed.

Can you please try adding this line to your program, before you call request.close():

request.headers.set("Host", "github.com");

This should make urls 5 and 6 work.

I tried the fix to set the Host header manually and then it works. So it seems that is the issue, and not the placement of the port. 

Regards, Robert 

On Friday, March 21, 2014 1:02:31 PM UTC+1, Robert Åkerblom-Andersson wrote:

[Søren Gjesse]

From our testing earlier today we found the Dart HTTP client is setting the Host header like this

host: github.com:443

That was causing a redirect response from github.com which we treated as a circular redirect as it was going back to where we just went. The fix mentioned above will override the Host header generated automatically. According to RFC 2616 section 14.23 (https://tools.ietf.org/html/rfc2616#section-14.23) it should okay to include the port. It only says what it means if the port is not there.

I assume using

  request.headers.set("Host", "github.com:443");

will not work.

However we so have a bug as we always remove port 80 regardless of the actual protocol, causing HTTPS on port 80 to result in a Host header without a port part (not a common case though).

Regards,

Søren

[Robert Åkerblom-Andersson]

Hi Sören,

Thanks for your comment and a good explanation. Your absolutely right, adding:

    request.headers.set("Host", "github.com:443");

Does not work and creates the redirection loop. We got it to work by overriding the header so that works for now. 

I read RFC 2616 section 14.23 now, and it feels like the real bug was/is at Github then. I will tell them about this.

Thanks for all the help in this thread today! Time to get some sleep...

Regards, Robert 

[Peter Ahé]

RFC 2818 (http://tools.ietf.org/html/rfc2818#section-2.3) says that:

"When HTTP/TLS is being run over a TCP/IP connection, the default port is 443."

RFC 2616 (https://tools.ietf.org/html/rfc2616#section-14.23) says that:

"A "host" without any trailing port information implies the default
port for the service requested (e.g., "80" for an HTTP URL)."

I read this as github.com implies github.com:443, so I'd argue that
the Dart implementation shouldn't need to append the port number.

I don't think that forwarding example.com:443 to example.com is
warranted by either RFC, and I'd argue this behavior is brittle.

But Dart's HTTP server implementation is not in a position to tell
anybody how to implement these RFCs. We simply need to run with what's
out there :-)

Cheers,
Peter


On Sat, Mar 22, 2014 at 2:55 AM, Robert Åkerblom-Andersson

[Robert Åkerblom-Andersson]

Hi Peter, I agree. 

Ivan from Github was very thankful for the feedback from the Dart community and said he was going to forward the feedback to the team!

Regards, Robert 

[Anders Johnsen]

Hi,

I've just landed a fix that removes ':443' from the host field, when the scheme is https.

Thanks for helping find this issue!

Cheers,

- Anders

[Robert Åkerblom-Andersson]

Hi Anders, 

Sounds great!

Cheers, Robert