Can't capture SOME iPhone requests

[Alessandro Ballini]

Hi everyone,

I have just installed Fiddler to capture some iPhone traffic. The process has been easy and the tool is very powerful.

I just can't understand why some (HTTPS) requests/responses are invisible in the sessions list.

This is the list of things I have already checked:

-HTTPS in installed correctly, as I can correctly see a lot of HTTPS requests and responses;

-there are no filters set up on Fiddler;

-all the settings are by default, I only tried to disable the caching;

-the traffic comes from a game, seems like I can see all the traffic from and to a couple of domains involved in the game, but see nothing on Fiddler when it's communicating with another domain;

-I disabled the mobile traffic and set up the Fiddler proxy on the iPhone, so all the traffic should be going through Fiddler.

How is this possibile? Why can't I see at all that slice of traffic to and from a domain?

Thank you in advance for your support.

[EricLaw]

Without more information, it won't be possible to answer your question. What is the application? Why do you think the traffic in question is sent over HTTP/HTTPS?

[Alessandro Ballini]

I am sorry Eric, I was at my first steps, now I have a deeper knowledge of what I am doing, so I can explain it better.

I am trying to debug an iPhone game traffic. The certificate has been successfully installed and I can correctly see a lot of traffic from browsers and other apps that use HTTPS. The Fiddler proxy is used for all the traffic though, I can see it passing through the laptop IP at port 8888, but a portion of it is not appearing on Fiddler.

I can imagine it's not exactly HTTPS traffic: on Wireshark I see TCP with TLSv1 encryption, in the encrypted TCP packets it's stating "Application: http" and the port between the proxy and the external server is 443.

I have tried exporting the private keys from Fiddler but I only get them for the traffic I actually see on Fiddler, so I have no .pem for this portion of traffic.

So my question is: can Fiddler help me to decrypt this portion of traffic or to retrieve the .pem or the master secret to feed Wireshark?
Please let me know if I can tell you anything else to better clarify the situation.
Thanks a lot for your great software and support.

[EricLaw]

I suspect that WireShark is merely guessing that the traffic is HTTPS based on the TLS handshake and the target port.

I don't understand what you mean when you say "I can see it passing through the laptop IP at port 8888" -- what do you see on Fiddler's log tab and in Fiddlers web sessions list?

[Alessandro Ballini]

I probably got it: that traffic is not going through the proxy, because the game is not completely honouring the iPhone network setting.

It's very difficult to compare Fiddler and Wireshark, because you have domain names on the first and IPs on the second and a lot of things are going on. I have tried comparing Wireshark log with the Fiddler proxy on to Wireshark log without Fiddler (sharing the internet connection) and in the second case I am seeing the traffic that is missing the first case, the traffic I am trying to decrypt.

Now, there is a way to get the iPhone non-proxied traffic on Fiddler too?

When sharing the laptop internet connection to the iPhone, I see everything on Wireshark, but nothing on Fiddler. Importing the sessions won't work because they are SSL crypted. I have tried playing with the proxy setting, but I could not have it going to Fiddler.

About this missing traffic, on Wireshark I see "TLSv1 Record Layer: Application Data Protocol: http". So it seems to be ok for the decryption, if only I could catch it. I have a Linux distribution too on my laptop, so I could use it if necessary to do the trick.

Thanks in adavance

[Alessandro Ballini]

I have managed to force all the traffic through Fiddler with Proxifier (very nice tool), so now I can see the CONNECT request on Fiddler, but the HTTPS stream is not starting.

Seems like the server is not sending the certificate in the response, but I cannot see anything wrong. All the browser HTTPS connections are working and Fiddler decrypts them. This game HTTPS traffic is not working with Proxifier/Fiddler, but is fine without the proxy.

This is a sample CONNECT, that is the only request I see, before the game times out saying that the servers are not working at the moment.

Any suggestion?

[EricLaw]

1> You've configured the device to trust Fiddler's certificate, right?

2> What, if anything do you see on Fiddler's Log tab?

3> Have you set the fiddler.network.https.SetCNFromSNI preference to True ?

4> What's the app?

[Alessandro Ballini]

I am tracking the PC game traffic now, since I found Proxifier to force it through the proxy, while the iOS and Android are not using the system proxy and I have not found a solution to force them through it yet.

I can successfully decrypt all the PC browser HTTPS traffic (and some from the game too) except this game stream. So yes, the certificate is in the Windows trusted.

I found this, I didn't notice before: 19:36:03:3037 !SecureClientPipeDirect failed: Authentication failed because the remote party has closed the transport stream. on pipe to (CN=XXXDOMAINNAMEXX, O=DO_NOT_TRUST_BC, OU=Created by http://www.fiddler2.com).

I have set the preference you said, with no luck.

[Alessandro Ballini]

This is an example of client and serve hello when I remove Fiddler.

Can I post something else to understand the difference?

[EricLaw]

Unfortunately, the client's use of SSLv3 instead of TLS1+SNI means that there's no way for Fiddler to automatically know what HTTPS certificate the client is expecting. Therefore, heuristics like the one you've used (e.g. looking at recent DNS requests) are probably the only reasonable approach.

The message "Authentication failed because the remote party has closed the transport stream" implies that the client was not happy with the certificate presented by Fiddler. It may be using Certificate Pinning, or it may expect a certificate containing some special properties not otherwise found in a server certificate.

[Alessandro Ballini]

Hi Eric,

I have found a sort of certificate mask inside the game, that I suppose being used to do the certificate pinning. The game is connecting to multiple hosts, so it's probably searching for partial matches.

Is there a way to have Fiddler use a certificate that is matching the mask below?

--
.rdata:03D12B00  2D 42 45 47 49 4E 20 43  45 52 54 49 46 49 43 41  -BEGIN CERTIFICA
.rdata:03D12B10  54 45 2D 2D 2D 2D 2D 00  01 00 00 00 03 00 00 00  TE-----. ... ...
.rdata:03D12B20  55 04 06 00 00 00 00 00  00 00 00 00 00 00 00 00  U .............
.rdata:03D12B30  03 00 00 00 03 00 00 00  55 04 07 00 00 00 00 00  ... ...U .....
.rdata:03D12B40  00 00 00 00 00 00 00 00  02 00 00 00 03 00 00 00  ........ ... ...
.rdata:03D12B50  55 04 08 00 00 00 00 00  00 00 00 00 00 00 00 00  U .............
.rdata:03D12B60  04 00 00 00 03 00 00 00  55 04 0A 00 00 00 00 00  ... ...U
.....
.rdata:03D12B70  00 00 00 00 00 00 00 00  05 00 00 00 03 00 00 00  ........ ... ...
.rdata:03D12B80  55 04 0B 00 00 00 00 00  00 00 00 00 00 00 00 00  U .............
.rdata:03D12B90  06 00 00 00 03 00 00 00  55 04 03 00 00 00 00 00  ... ...U .....
.rdata:03D12BA0  00 00 00 00 00 00 00 00  07 00 00 00 03 00 00 00  ........ ... ...
.rdata:03D12BB0  55 1D 11 00 00 00 00 00  00 00 00 00 00 00 00 00  U .............
.rdata:03D12BC0  08 00 00 00 03 00 00 00  55 1D 13 00 00 00 00 00  ... ...U .....
.rdata:03D12BD0  00 00 00 00 00 00 00 00  09 00 00 00 09 00 00 00  ........    ...    ...
.rdata:03D12BE0  2A 86 48 86 F7 0D 01 01  01 00 00 00 00 00 00 00  *ÆHÆ¢ .......
.rdata:03D12BF0  0A 00 00 00 09 00 00 00  2A 86 48 86 F7 0D 01 01 
...    ...*ÆHÆ¢
.rdata:03D12C00  02 00 00 00 00 00 00 00  0B 00 00 00 09 00 00 00  ....... ...    ...
.rdata:03D12C10  2A 86 48 86 F7 0D 01 01  04 00 00 00 00 00 00 00  *ÆHÆ¢ .......
.rdata:03D12C20  0C 00 00 00 09 00 00 00  2A 86 48 86 F7 0D 01 01  ...    ...*ÆHÆ¢
.rdata:03D12C30  05 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ...............
.rdata:03D12C40  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
.rdata:03D12C50  92 75 A1 5B 08 02 40 B8  9B 40 2F D5 9C 71 C4 51  Òuá[ @¬Û@/-Üq-Q
.rdata:03D12C60  58 71 D8 F0 2D 93 7F D3  0C 8B 1C 7D F9 2A 04 86  Xq+¨-Ó LË }•* Æ
.rdata:03D12C70  F1 90 D1 31 0A CB D8 D4  14 12 90 3B 35 6A 06 51  ¸ÐT1
T+L¶ Ð;5j Q
.rdata:03D12C80  49 4C C5 75 EE 0A 46 29  80 F0 D5 3A 51 BA 5D 6A  IL+uþ
F)À¨-:Q¦]j
.rdata:03D12C90  19 37 33 43 68 25 2D FE  DF 95 26 36 7C 43 64 F1  73Ch%-¦-Õ&6|Cd¸
.rdata:03D12CA0  56 17 0E F1 67 D5 69 54  20 FB 3A 55 93 5D D4 97  V ¸g-iT v:UÓ]L×
.rdata:03D12CB0  BC 3A D5 8F D2 44 C5 9A  FF CD 0C 31 DB 9D 94 7C  -:-ÏTD+Ú =1-ÝÔ|
.rdata:03D12CC0  A6 66 66 FB 4B A7 5E F8  64 4E 28 B1 A6 B8 73 95  æffvKç^°dN(-æ¬sÕ
.rdata:03D12CD0  C2 0D 06 A5 D7 11 AC E9  BB 31 05 2C 82 50 DC 6F  T å+ ìù¬1 ,ÂP-o
.rdata:03D12CE0  B9 78 81 93 1E 1D 70 A0  73 16 D5 FB 89 9A CF 67  ¦xÁÓ pàs -vÉÚ¦g
.rdata:03D12CF0  00 03 64 C6 14 DD 17 1F  2E 8E 33 B3 5C 3C 3B 62  . d¦¶¦ .Î3¦\<;b
.rdata:03D12D00  20 2E B3 6A CB D8 72 C8  55 E6 B2 C6 65 A3 03 35   .¦jT+rLUö-¦eã 5
.rdata:03D12D10  F9 16 3C 4B 78 74 AD 3A  08 52 69 F8 A5 F3 89 78  • <Kxtí: Ri°åºÉx
.rdata:03D12D20  38 16 62 C3 ED E2 01 32  BC 6E EB D9 B8 FC 3F C7  8 b+ýò 2-nû-¬¹?¦
.rdata:03D12D30  EE F2 E8 EC 1B 2B A8 B8  B1 3F A1 18 71 76 17 AE  þªøü +è¬-?á qv î
.rdata:03D12D40  61 A4 3E EA E8 73 19 98  9F 7E C8 6E 3A 86 09 75  aä>úøs Øß~Ln:Æ    u
.rdata:03D12D50  2E 7E B5 A3 6E 7A 14 8B  7B 81 D0 22 5C 6D 01 EF  .~¦ãnz¶Ë{Á¦"\m ÿ
.rdata:03D12D60  8D 88 6C 89 61 12 5C 68  32 67 C2 E6 8B AD F3 EF  ÍÈlÉa \h2gTöËíºÿ
.rdata:03D12D70  02 FC 86 DE DA 47 BD F8  B8 55 BB 75 E6 40 34 8B  ¹Æ¦-G-°¬U¬uö@4Ë
.rdata:03D12D80  8E C6 15 9C 59 AF 73 C0  83 CE 45 83 BB E0 EB 2C  Φ§ÜYïsLÃ+Eìðû,
.rdata:03D12D90  59 9B 79 DC 56 36 74 28  82 8B E3 C3 85 29 10 D6  YÛy-V6t(ÂËó+Å) ã
.rdata:03D12DA0  D0 65 E5 62 93 49 22 BB  0C 4B 3C 75 8C 97 DF 87  ¦eõbÓI"¬K<uÌ×-Ç
.rdata:03D12DB0  0A 13 CA E9 4B 5B 11 E5  29 29 73 D8 11 E1 91 7C 
¦ùK[ õ))s+ ñÑ|
.rdata:03D12DC0  3B 2D D7 DC 8D FD CD 87  0E 68 62 77 6D AA D9 01  ;-+-ͤ=Ç hbwmê-
.rdata:03D12DD0  95 C3 21 12 8E 40 C5 0D  01 5F 76 5E 66 94 D9 73  Õ+! Î@+ _v^fÔ-s
.rdata:03D12DE0  2C 58 19 22 B8 C9 FC 7A  39 90 2A 77 72 7C 1D 3E  ,X "¬ã¹z9Ð*wr| >
.rdata:03D12DF0  F7 D8 55 E3 AF 42 CB 87  30 02 DC 5B AC 70 E6 B8  ¢+UóïBTÇ0 -[ìpö¬
.rdata:03D12E00  44 B4 2B 35 EB 93 D2 17  05 7E CB 46 D6 5C 53 A0  D++5ûÓT ~TFã\Sà
.rdata:03D12E10  32 51 9D 74 64 58 F9 0C  9A 00 EA 5E 44 49 64 72  2QÝtdX•Ú.ú^DIdr
.rdata:03D12E20  F4 CD 10 E2 85 0A F9 34  EE B3 88 66 A9 A5 A4 5A  ¯= òÅ
•4þ¦ÈféåäZ
.rdata:03D12E30  D0 0E 98 7F 58 0D 2B 52  BB 86 A9 7E 2E FA B2 48  ¦ Ø X +R¬Æé~.·-H
.rdata:03D12E40  7C 8D DB 2D 5F 01 75 A2  8D 06 3B 8B B4 61 07 C9  |Í--_ uâÍ ;Ë+a ã
.rdata:03D12E50  BE 22 99 F8 1B D1 B5 57  66 04 4D 35 F4 91 71 96  -"Ù° T¦Wf M5¯ÑqÖ
.rdata:03D12E60  B5 99 08 25 9B 97 C8 3A  F3 20 B1 DD 9E 98 0C 4A  ¦Ù %Û×L:º -¦ÞØJ
.rdata:03D12E70  63 B7 A6 CE B0 01 CE F8  93 6A F3 0C 6E 9F B1 E9  c¬æ+- +°Ójºnß-ù
.rdata:03D12E80  84 7B 81 98 41 E6 81 DC  3D 2C E7 B4 6B E3 9E FC  Ä{ÁØAöÁ-=,÷+kóÞ¹
.rdata:03D12E90  08 16 D7 B3 D5 B9 66 12  99 7C 6D 71 C8 4D BE C7  +¦-¦f Ù|mqLM-¦
.rdata:03D12EA0  0F E3 FB 37 AD D5 75 87  21 6B 86 D0 44 14 5A 54  ¤óv7í-uÇ!kƦD¶ZT
.rdata:03D12EB0  79 39 96 69 56 C9 B9 31  CD 89 61 58 E1 D9 76 05  y9ÖiVã¦1=ÉaXñ-v
.rdata:03D12EC0  05 AD F7 B9 02 AF A7 FD  47 91 A2 22 34 5A 31 D1  í¢¦ ïç¤GÑâ"4Z1T
.rdata:03D12ED0  AF 24 08 08 29 7A 35 9E  60 0C AA E7 4B 3B 4E DC  ï$ )z5Þ`ê÷K;N-
.rdata:03D12EE0  7C BC 3C 45 1C BB 2B E0  FE 29 02 F9 57 08 A3 64  |-<E ¬+ð¦) •W ãd
.rdata:03D12EF0  85 15 27 F5 F1 AD C8 31  89 5D 22 E8 2A AA A6 42  ŧ'¿¸íL1É]"ø*êæB
.rdata:03D12F00  B3 8F F8 B9 55 B7 B1 B7  4B B3 FE 8F 7E 07 57 EC  ¦Ï°¦U¬-¬K¦¦Ï~ Wü
.rdata:03D12F10  EF 43 DB 66 62 15 61 CF  60 0D A4 D8 DE F8 E0 C3  ÿC-fb§a¦` ä+¦°ð+
.rdata:03D12F20  62 08 3D 54 13 EB 49 CA  59 54 85 26 E5 2B 8F 1B  b =T ûI¦YTÅ&õ+Ï
.rdata:03D12F30  9F EB F5 A1 91 C2 33 49  D8 43 63 6A 52 4B D2 8F  ßû¿áÑT3I+CcjRKTÏ
.rdata:03D12F40  E8 70 51 4D D1 89 69 7B  C7 70 F6 B3 DC 12 74 DB  øpQMTÉi{¦p¡¦- t-
.rdata:03D12F50  7B 5D 4B 56 D3 96 BF 15  77 A1 B0 F4 A2 25 F2 AF  {]KVLÖ¬§wá-¯â%ªï
.rdata:03D12F60  1C 92 67 18 E5 F4 06 04  EF 90 B9 E4 00 E4 DD 3A  Òg õ¯ ÿЦô.ô¦:
.rdata:03D12F70  B5 19 FF 02 BA F4 3C EE  E0 8B EB 37 8B EC F4 D7  ¦   ¦¯<þðËû7Ëü¯+
.rdata:03D12F80  AC F2 F6 F0 3D AF DD 75  91 33 19 1D 1C 40 CB 74  쪡¨=ï¦uÑ3 @Tt
.rdata:03D12F90  24 19 21 93 D9 14 FE AC  2A 52 C7 8F D5 04 49 E4  $ !Ó-¶¦ì*R¦Ï- Iô
.rdata:03D12FA0  8D 63 47 88 3C 69 83 CB  FE 47 BD 2B 7E 4F C5 95  ÍcGÈ<iÃT¦G-+~O+Õ
.rdata:03D12FB0  AE 0E 9D D4 D1 43 C0 67  73 E3 14 08 7E E5 3F 9F  î ÝLTCLgsó¶ ~õ?ß
.rdata:03D12FC0  73 B8 33 0A CF 5D 3F 34  87 96 8A EE 53 E8 25 15  s¬3
¦]?4ÇÖÊþSø%§
.rdata:03D12FD0  90 B3 80 C1 E4 E5 46 AD  70 60 3D BA E5 14 DD 9E  ЦÀ+ôõFíp`=¦õ¶¦Þ
.rdata:03D12FE0  8A 5E 8B 75 5A E6 CA 6D  41 A5 23 E8 39 85 26 7A  Ê^ËuZö¦mAå#ø9Å&z
.rdata:03D12FF0  A7 55 77 9A 48 A1 92 7E  3A 1E 1A F1 27 AB A3 4C  çUwÚHáÒ~: ¸'ëãL
.rdata:03D13000  39 CC CB 3D 47 AF 81 AE  16 6A 5C 37 EF 45 41 FD  9¦T=GïÁî j\7ÿEA¤
.rdata:03D13010  FB 9A 97 3C A0 43 9D C6  DF 17 21 D1 8A A2 56 C2  vÚ×<àCݦ- !TÊâVT
.rdata:03D13020  03 49 84 12 81 3E C9 0A  54 60 66 B9 8C 54 E4 F9  IÄ Á>ã
T`f¦ÌTô•
.rdata:03D13030  E6 F9 94 F1 E0 5F 75 11  F2 29 B9 E4 86 A2 B1 89  ö•Ô¸ð_u ª)¦ôÆâ-É
.rdata:03D13040  AD A6 1E 83 29 63 B2 F0  54 1C 85 0B 7A E7 E1 2E  íæ Ã)c-¨T Å z÷ñ.
.rdata:03D13050  0D AF A4 BD CD E7 B1 5A  D7 8C 05 5A 0E 4B 73 28   ïä-=÷-Z+Ì Z Ks(
.rdata:03D13060  8B 75 5D 34 D8 77 0B E1  74 62 E2 71 30 62 D8 BC  Ëu]4+w ñtbòq0b+-
.rdata:03D13070  8A 05 E5 31 63 4A 54 89  6A 33 78 A7 4E 55 24 1D  Ê õ1cJTÉj3xçNU$
.rdata:03D13080  97 EF 1A E4 12 C6 0F 30  18 B4 34 4D E1 D8 23 3B  ×ÿ ô ¦¤0 +4Mñ+#;
.rdata:03D13090  21 5B 2D 30 19 25 0E 74  F7 A4 21 4B A0 A4 20 C9  ![-0 % t¢ä!Kàä ã
.rdata:03D130A0  6C CD 98 56 C0 F2 A8 5F  3E 26 75 A0 0D F8 36 88  l=ØVLªè_>&uà °6È
.rdata:03D130B0  8A 2C 5A 7D 67 30 A9 0F  D1 99 70 2E 78 E1 51 26  Ê,Z}g0é¤TÙp.xñQ&
.rdata:03D130C0  AF 55 7A 24 BE 8C 39 0D  77 9D DE 02 C3 0C BD 1F  ïUz$-Ì9 wݦ +-
.rdata:03D130D0  2D 2D 2D 2D 2D 45 4E 44  20 43 45 52 54 49 46 49  -----END CERTIFI
.rdata:03D130E0  43 41 54 45 2D 2D 2D 2D  2D 00 00 00 2D 2D 2D 2D  CATE-----...----
.rdata:03D130F0  2D 42 45 47 49 4E 20 58  35 30 39 20 43 45 52 54  -BEGIN X509 CERT
.rdata:03D13100  49 46 49 43 41 54 45 2D  2D 2D 2D 2D 00 00 00 00  IFICATE-----....
.rdata:03D13110  2D 2D 2D 2D 2D 45 4E 44  20 58 35 30 39 20 43 45  -----END X509 CE
.rdata:03D13120  52 54 49 46 49 43 41 54  45 2D 2D 2D 2D 2D 00 00  RTIFICATE----