Mozilla Cloud non-Decryptable Download?

[miro....@croatiafidelis.hr]

Hi!

I've recently discovered and have been using Mozilla's NSS, gaining lots of understanding, lots of insight into what really happened in various conversations when I go online. The only way, however costly, for me to go, as I have had undeniable clashes with censorship and intrusions. [1]

I have recently stumbled upon what appears to be a download into my machine from a Mozilla Cloud.

Pls. note upfront that I don't claim anything is wrong there, and I don't claim all is right there either. My knowledge is insufficient for me to be able to fathom all the details.

However, if all else is mostly decryptable, why is this not?

What info is needed to decrypt what that download is, where can I find that info?

I have posted about it at:


Mozilla Cloud non-Decryptable Download? (the title to be)
https://forums.gentoo.org/viewtopic-t-1031758.html

so I'm just repasting these two lines:

> You can check all with the traffic dump in the dir:

> http://www.croatiafidelis.hr/foss/cap/cap-151029-MozCloud/

and my understanding of it I have deployed in the Gentoo Forums' topic above.

I'll be greateful for any clues helpful to my decrpyting of that download.

Regards!

--
Miroslav Rovis
Zagreb, Croatia
http://www.CroatiaFidelis.hr

---
[1] If you're interested in those, pls. see my other topic:

SSL Decode & My Hard-Earned Advice for SPDY/HTTP2 in Firefox
https://forums.gentoo.org/viewtopic-t-1029408.html

where I cared to link to such clashes in the opening posts (as well as later).

[Julien Vehent]

On Sun 1.Nov'15 at 13:22:25 +0100, miro....@croatiafidelis.hr wrote:
> I'll be greateful for any clues helpful to my decrpyting of that download.

Your PCAP indicates the source IP is a cloudfront endpoint, which we use
to distribute updates, tracking protection, addons and so on. None of
those binaries are encrypted, only the transport is secured with TLS, so
if you get the cleartext traffic you'll obtain the cleartext binary.

- Julien

[miro....@croatiafidelis.hr]

Hi Julien!

Opening my PCAP, and pasting in this filter:

"(ip.addr==54.192.55.37) || (ip.addr==54.192.12.211) ||
(ip.addr==216.137.59.141)"
(one line, without quotes)

and then:

File > "Export Specified Packets". "Packet Range" is "All Packets", the
"Displayed" is selected already, and saving it as:

dump_151029_1757_g0n_MozCloud.pcap

gets, of all the conversations, just the conversations with the Mozilla
Clouds hosts, as I explained it in:

https://forums.gentoo.org/viewtopic-t-1031758.html#7835156

Then we can concentrate solely on those, without all the plethore of
other hosts interfereing in the analysis, so to speak (only 8 tcp
streams now, 0-7).

And then, as I explain in the:

https://forums.gentoo.org/viewtopic-t-1031758.html#7835158

it's easy to get the tcp.stream eq $i (where $i is 0-7) out.

And then, it's easy to get to the problem, by saving "tcp.stream eq 5",
the 6th tcp stream (not SLL, but plain TCP stream) into a file.

It's:

-rw-r--r-- 1 miro miro  72M 2015-10-30 16:45 dump_151029_1757_g0n_MozCloud.pcap

I really explained it in that topic on Gentoo Forums, but out of respect
for you, I'm repeating some of the explanations, to just show to you
Mozilla devs that I'm talking verifiable facts here (in case some of you
have skimmed too quickly through this issue)...

Once the tcp strem 5 is saved you get:

$ ls -l  dump_151029_1757_g0n_MozCloud_s5.dump
-rw-r--r-- 1 miro miro 67764933 2015-10-30 22:41 dump_151029_1757_g0n_MozCloud_s5.dump
$ ls -lh  dump_151029_1757_g0n_MozCloud_s5.dump
-rw-r--r-- 1 miro miro 65M 2015-10-30 22:41 dump_151029_1757_g0n_MozCloud_s5.dump
$

That's 65M of data, that ought to not be encrypted, as you say, and I
accept your word that it ought not be encrypted.

But it is, as it appears to me. Or, I must leave that window of
possibility, as I've only discovered the PFS conversations can be
decrypted very recently... And I thank you, Mozilla devs for the
Network Security Services! Or, I must leave that window of possibility,
it's the lack of my knowledge, but it can be decrypted.

In that case, however, the keys or something, has to be around in the
PCAP, else it's an intrusion on my, the user, if that missing engredient
(or what to call it) is hidden from me, or if that missing engredient,
is in Mozilla Cloud databases only, and not available to me, the user,
in whose machine the download happened.

Because, the dump_151029_1757_g0n_MozCloud_s5.dump can be taken the
gzip'd parts out of it, and they qualify as gzip, on the outside, the
Unix's file command says of them (but pls. see the

https://forums.gentoo.org/viewtopic-t-1031758.html#7835158
for detailed analysis how I took those gzip'd files out of the dump with
hexedit)...

But, the gzip'd files from that dump show, just the third one:

$ file dump_151029_1757_g0n_MozCloud_s5_03.gz
dump_151029_1757_g0n_MozCloud_s5_03.gz: gzip compressed data, ASCII,
extra field, encrypted
$

and then they can't be gunzipped:

$ gunzip dump_151029_1757_g0n_MozCloud_s5_03.gz
gzip: dump_151029_1757_g0n_MozCloud_s5_03.gz is encrypted -- not
supported
$

So, where are the ingredients to gunzip that file?

Only in Mozilla databases, or somewhere in the PCAP?

If in the PCAP, where?

And if in Mozilla databases, can you pls. send them to me?

Thank you in advance!

[Julien Vehent]

On Sun 1.Nov'15 at 17:19:59 +0100, miro....@croatiafidelis.hr wrote:
> So, where are the ingredients to gunzip that file?

Gzip compression is a standard feature of HTTP. The data is simply
compressed by the web servers that serve it to Firefox.

I'm not familiar with gzip's internal, but I can only imagine that the
"encryption" error is a misreading of the file. Maybe you're missing
chunks? Maybe your extraction process corrupted the data?

Again, we don't encrypt binary files.

- Julien

[miro....@croatiafidelis.hr]

On 151101-13:31-0500, Julien Vehent wrote:
> On Sun 1.Nov'15 at 17:19:59 +0100, miro....@croatiafidelis.hr wrote:
> > So, where are the ingredients to gunzip that file?
>
> Gzip compression is a standard feature of HTTP.

I'm familiar with that. I gunzip'd lots of data from tcp stream.

> The data is simply
> compressed by the web servers that serve it to Firefox.

That's generally the case, sure. I know that. But I'm not talking
generally here. I have posted what anybody in the world, and esp.
Mozilla devs with complete ease, can check. Not repeating. Gave all the
links. Even repeated the explanation. Why did you cut it out?

Yes why did you cut it out and simply talk generally what I know, and
what even beginners know?

Other readers, pls. there is no need to reply to this talk here. I am
replying out of courtesy, not out of need.

Pls reply to my previous emails, esp. my immediately previous email to
this email, the one that Julian cut out the real information out, or, if
the link will show correctly (as it does no on phpBB-driven Gentoo
Forums, where the link must be pasted into your browser's address bar,
(as I explained in
https://forums.gentoo.org/viewtopic-t-1031758.html#7835158 calling Goog
the rapist of standars; sorry!, I had to)...

Reply to, if the link will show correctly (sending with the good and
honest Mutt):

https://groups.google.com/d/msg/mozilla.dev.security/abSHPU4EaP8/Tyd_nEh5CAAJ

please. Thank you!

>
> I'm not familiar with gzip's internal, but I can only imagine that the
> "encryption" error is a misreading of the file. Maybe you're missing
> chunks? Maybe your extraction process corrupted the data?
>

I'll give you the margin of error possibility, here.

But, to me, it does not look like a gzip error. I think gzip is
reporting what is sees, gzip is stating what is examined and found out
for being the case.

But the thing is, those files are verifiable that they happened to me,
to my machine at that time, and those procedures that I described can
easily be repeated. So, while...

(

> Again, we don't encrypt binary files.
>

I take you word that it is against the prescibed procedure at Mozilla.
And I aplaud Mozilla for that prescribed procedure... Just has, maybe,
that procedure here been violated?

> - Julien
)

[So, while] I can take that (the choic that you don't deal with gzip
much), and thank you anyway... I still hope not all Mozilla devs are
unfamiliar with gzip.

Anybody can share more light on this one.

And, again, you don't need to reply to this email, but rather to the
immediately previous mail of mine, and pls reply with checking up those
procedures after which the file is there, just as I explained in that
email of mine previous to this one, and, to which the Unix's file and
gzip command state clearly what they found about that file.

For which I will be thankful to you!

[Andrew Sutherland]

On Sun, Nov 1, 2015, at 07:22 AM, miro....@croatiafidelis.hr wrote:
> I'll be greateful for any clues helpful to my decrpyting of that
> download.

It's likely a MAR file. If it is, the file header should start with the
magic keyword "MAR1". See https://wiki.mozilla.org/Software_Update:MAR
for more details and a link to a file at the bottom that understands
them.

You make reference to a "GET" inside the HTTPS data-stream. It may be
worth manually downloading that URL separately using wget/curl/other and
seeing what is extracted from that. It's possible that in the
hexediting you extracted part of the framing around the file, and this
would likely help shed light on that. Alternately, from brief
searching, it sounds like wireshark should already be able to save the
decompressed content streams from HTTP via "File -> Export Objects ->
HTTP"?

Andrew

[miro....@croatiafidelis.hr]

Hi!

This looks precious to me. Feels right away as the honest advice:

> _______________________________________________
> dev-security mailing list
> dev-se...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security

Now, pls. here comes in my turtle speed of doing things. I've only
reached this far to even discuss these issues, because of huge work that
I input.

Your advice is accepted here with sincere gratitude, but for me to fully
understand it, it will take time!

I will be glad if the download (it's an autodownload, behind what I was
doing, not initiated by me in any way, which can be seen from the
screencast associated (taken with my uncenz (primitive) program:

https://github.com/miroR/uncenz

Thank you, Andrew! You look like what I called Old Guard Mozilla (in the
previous topic on Gentoo, linked in the outset of this topic in Gentoo,
which bears the same title as this thread!

If only Old Guard Mozilla folks would lead in your community, and if
Mozilla left the Octopus of the Internet, the Goog, alone!

[miro....@croatiafidelis.hr]

I meant to say:

On 151101-21:54+0100, miro....@croatiafidelis.hr wrote:
> I will be glad if the download (it's an autodownload, behind what I was
> doing, not initiated by me in any way, which can be seen from the
> screencast associated (taken with my uncenz (primitive) program:
>
> https://github.com/miroR/uncenz
>

) [I will be glad if the download] was legitimate.

[miro....@croatiafidelis.hr]

Hi!

The concrete 70M non-decryptable from the previous topic (on Gentoo
Forums, and in this same thread here) I have not yet solved.

Because I have more urgent issues in my use of Firefox, and because of
the turtle-speed of my progress in learning this knowhow.

On 151101-21:54+0100, miro....@croatiafidelis.hr wrote:

> Hi!
>
> This looks precious to me. Feels right away as the honest advice:
>
> On 151101-15:27-0500, Andrew Sutherland wrote:
> > On Sun, Nov 1, 2015, at 07:22 AM, miro....@croatiafidelis.hr wrote:
> > > I'll be greateful for any clues helpful to my decrpyting of that
> > > download.
> >
> > It's likely a MAR file. If it is, the file header should start with the
> > magic keyword "MAR1". See https://wiki.mozilla.org/Software_Update:MAR
> > for more details and a link to a file at the bottom that understands
> > them.
> >
> > You make reference to a "GET" inside the HTTPS data-stream. It may be
> > worth manually downloading that URL separately using wget/curl/other and
> > seeing what is extracted from that. It's possible that in the
> > hexediting you extracted part of the framing around the file, and this
> > would likely help shed light on that. Alternately, from brief
> > searching, it sounds like wireshark should already be able to save the
> > decompressed content streams from HTTP via "File -> Export Objects ->
> > HTTP"?
> >
> > Andrew
>

...

> Your advice is accepted here with sincere gratitude, but for me to fully
> understand it, it will take time!
>

And by now I have understood the most of the advice above. And thanks
again! (And really sorry that I had to postpone the completion of my
analysis on that 70M non-decryptable.)

But it looks to me other non-decryptables seem to feature in a new
miniature screencast/traffic-dump pair at:

http://www.croatiafidelis.hr/foss/cap/cap-151125-plus-cert/

I have tried to very carefully analyze esp. the dump, and, in very
brief, in a matter of a minute or two, I believe an expert can see from
the dump of 617K only:

dump_151125_1447_g0n.pcap

that the "tcp.stream eq 14" filter entered, and decrypted SSL saved (the
dump_151125_1447_g0n_sesskeys.txt, that is the session keys, of course,
previously deployed, which is there to download), the two files, both
from:

https://tracking-protection.cdn.mozilla.net/mozstd-track-digest256/

and downloading (at least yesterday was such the case) exactly as they
can be gotten as exported objects from the dump, so these were
downloading and are exported as:

total 360
-rw-r--r-- 1 miro miro 54294 2015-10-22 00:07 1445465225
-rw-r--r-- 1 miro miro 307799 2015-11-03 00:37 1446507423
total 360K
-rw-r--r-- 1 miro miro 54K 2015-10-22 00:07 1445465225
-rw-r--r-- 1 miro miro 301K 2015-11-03 00:37 1446507423

It can be seen that they are not MOZ1; again, when SSL decrypted (no
MOZ1 to be found with hexedit).

I have analyzed more in detail (but that is also too extensively
detailed for this list, it's for the forumers there to also understand),
and posted more questions what these files could be, at:

More non-Decryptables (from Mozilla Cloud)
https://forums.gentoo.org/viewtopic-t-1034140.html

where if you read, you can skip the first (for-newbies) post, and skim
only through the second post, if in a hurry:
https://forums.gentoo.org/viewtopic-t-1034140.html#7847334

But in short: if these are legitimate, what are they, and where are they
now in my system.

(Also why any "tracking protection" when I don't have it in my Firefox
settings?)

Thanks if I get info what these two (small really) files exactly are!

[Andrew Sutherland]

On Fri, Nov 27, 2015, at 06:46 AM, miro....@croatiafidelis.hr wrote:
> previously deployed, which is there to download), the two files, both
> from:
>
> https://tracking-protection.cdn.mozilla.net/mozstd-track-digest256/

> It can be seen that they are not MOZ1; again, when SSL decrypted (no
> MOZ1 to be found with hexedit).

See
https://feeding.cloud.geek.nz/posts/how-tracking-protection-works-in-firefox/
for a description of the underlying tracking protection implementation
and links to a script that can help you dump and analyze these files.
(And links to other info too.)

Andrew

[Francois Marier]

On 27/11/15 03:46 AM, miro....@croatiafidelis.hr wrote:
> https://forums.gentoo.org/viewtopic-t-1034140.html#7847334
>
> But in short: if these are legitimate, what are they, and where are they
> now in my system.

The small one is the tracking protection blacklist
(mozstd-track-digest256) and the larger one is the entity whitelist
(mozstd-trackwhite-digest256).

You can find more info about these lists here:

https://wiki.mozilla.org/Security/Tracking_protection#Lists

> (Also why any "tracking protection" when I don't have it in my Firefox
> settings?)

They are downloaded if one (or both) of these prefs is turned on:

privacy.trackingprotection.enabled (default: false)
privacy.trackingprotection.pbmode.enabled (default: true)

Francois

[miro....@croatiafidelis.hr]

And I also confirm that I have received kind reply from Andrew
Sutherland, who wrote (pasting):

See
https://feeding.cloud.geek.nz/posts/how-tracking-protection-works-in-firefox/
for a description of the underlying tracking protection implementation
and links to a script that can help you dump and analyze these files.
(And links to other info too.)

(pasted).

I will study those instructions. Just, I work only at turtle-speed no
matter how I tried ;-) .

Thanx!

On 151127-15:59-0800, Francois Marier wrote:
> On 27/11/15 03:46 AM, miro....@croatiafidelis.hr wrote:

> > https://forums.gentoo.org/viewtopic-t-1034140.html#7847334
> >
> > But in short: if these are legitimate, what are they, and where are they
> > now in my system.
>

> The small one is the tracking protection blacklist
> (mozstd-track-digest256) and the larger one is the entity whitelist
> (mozstd-trackwhite-digest256).
>
> You can find more info about these lists here:
>
> https://wiki.mozilla.org/Security/Tracking_protection#Lists

I will be try and carefully ponder over that.

> > (Also why any "tracking protection" when I don't have it in my Firefox
> > settings?)
>

> They are downloaded if one (or both) of these prefs is turned on:
>
> privacy.trackingprotection.enabled (default: false)
> privacy.trackingprotection.pbmode.enabled (default: true)
>
> Francois

And I will check these.

Thanx.

And, somewhat important, and really confusing, my Firefox, with which I
was able to (mis)report a bug at:

tshark saves raw stream in ascii file, content unrecoverable
https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=11750

(just to show it should work in the case below: that was closest to the
last previous use of Firefox, 6 days ago now; and see what happens to me
below, for why I can't use it before I get the command of it myself)

[really confusing,] ... [my Firefox] does not do any GET'ing nor
POST'ing when it should, as per:

http://croatiafidelis.hr/foss/cap/cap-151127-plus/

All there, not yet worked enough on it to make proper questions, other
then: no GET's, no POST's so how could I have logged in?

Made with my (primitive) program https://github.com/miroR/uncenz so the
screencast and the traffic dump are corresponding.

Questions not proper yet, but if anyone can help tell the reason for my
not logging in there, I'll be greatful.

Regards!