My bootrom disassembly

[Tom Trebisky]

On and off over the past year I have been busy disassembling the ESP8266 bootrom.

I am back in the thick of it again lately and have been learning new things,

and updating my annotated copy on github regularly.

I am aware of Paul's "ScratchAbit", but I started this project over a year ago and have gotten familiar enough

with the xtensa assembly code that I can just keep going like I am.

Anyway, I figured I should make others aware of what I am up to, as well as to get in touch with this

group.  It may help me or others avoid duplication of effort.

https://github.com/trebisky/esp8266/tree/master/reverse/bootrom

What most of my effort goes into is in studying the code, and then making

annotations in the form of comments to document what I have learned.

    Tom

[Peter Korsgaard]

On Fri, Mar 3, 2017 at 11:44 PM, Tom Trebisky <t...@mmto.org> wrote:
> On and off over the past year I have been busy disassembling the ESP8266
> bootrom.
> I am back in the thick of it again lately and have been learning new things,
> and updating my annotated copy on github regularly.
>
> I am aware of Paul's "ScratchAbit", but I started this project over a year
> ago and have gotten familiar enough
> with the xtensa assembly code that I can just keep going like I am.
>
> Anyway, I figured I should make others aware of what I am up to, as well as
> to get in touch with this
> group. It may help me or others avoid duplication of effort.
>
> https://github.com/trebisky/esp8266/tree/master/reverse/bootrom

Thanks, this looks very interesting!

--
Bye, Peter Korsgaard

[Alex Stewart]

I just realized I apparently missed Tom's original email a couple of weeks ago (thanks to Peter for replying to it so I actually saw it :) )

Looking at what you've already done and obviously gone through to get there, I'm realizing it's probably a bit late now, but FYI, I created a much more sophisticated Xtensa disassembler of my own to fix a lot of the deficiencies with the existing tools:  https://bitbucket.org/foogod/xtobjdis

It was originally written to disassemble the object files of the Espressif SDK, but it was later extended to also work with raw binary dumps (such as the boot ROM)..  If anyone wants to use it for that, you'll want to use a "fixups" file to give it hints about where various things are located within the binary blob (there's a (I think) fairly complete fixups file for use with the ESP8266 boot ROM available in my esp8266-re-tools repo (https://bitbucket.org/foogod/esp8266-re-tools) under the "fixups" directory)..

You may also want to add a link to your work from the ESP8266-RE Wiki (http://esp8266-re.foogod.com/) as well..

--
You received this message because you are subscribed to the Google Groups "esp8266-re" group.
To unsubscribe from this group and stop receiving emails from it, send an email to esp8266-re+unsubscribe@googlegroups.com.
To post to this group, send email to esp82...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/esp8266-re/CACXmViY4yYuKbcwcOQjqUN6%2B05g6UNaWY1zf6Ta%2BChdBbNLQcw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Tom Trebisky]

Ok.  Good.  I just didnt want to be working in a vacuum.

You received this message because you are subscribed to a topic in the Google Groups "esp8266-re" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/esp8266-re/X3uJ0MGBhj4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to esp8266-re+unsubscribe@googlegroups.com.

To post to this group, send email to esp82...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/esp8266-re/CACxLcFd%3D2sJRPVg0cyQ%2BeNup-z3RcYj6dqQeYqWmaPm%3D%2Bn1JwQ%40mail.gmail.com.

[Tom Trebisky]

Hi Alex,

I just got back from a trip with my son Alex and am catching up on emails.  I appreciate hearing from you.

So, do I have permission to add links and/or other content to the Wiki?

You received this message because you are subscribed to a topic in the Google Groups "esp8266-re" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/esp8266-re/X3uJ0MGBhj4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to esp8266-re+...@googlegroups.com.

To post to this group, send email to esp82...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/esp8266-re/CACxLcFd%3D2sJRPVg0cyQ%2BeNup-z3RcYj6dqQeYqWmaPm%3D%2Bn1JwQ%40mail.gmail.com.

[Alex Stewart]

Of course!  It's a public wiki (it requires registration just to keep spammers at bay, but you can set up an account for yourself fairly easily and then will have full write access).  Everyone is welcome to contribute anything that might be useful to fellow reverse-engineers.

To unsubscribe from this group and all its topics, send an email to esp8266-re+unsubscribe@googlegroups.com.

To post to this group, send email to esp82...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/esp8266-re/CACxLcFd%3D2sJRPVg0cyQ%2BeNup-z3RcYj6dqQeYqWmaPm%3D%2Bn1JwQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "esp8266-re" group.
To unsubscribe from this group and stop receiving emails from it, send an email to esp8266-re+unsubscribe@googlegroups.com.
To post to this group, send email to esp82...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/esp8266-re/4567a360-9a17-f987-682c-30d030734004%40mmto.org.

[Tom Trebisky]

Thanks Alex,

I have veered away from ESP8266 temporarily.  Now focusing on a little ARM based board called the Orange Pi.
Working up an ethernet driver for it.  But I'll be back. ......  I've been meaning to participate more on the forum.
Thanks for the reply.

[Peter Korsgaard]

Hello,

On Fri, Mar 24, 2017 at 6:13 PM, Tom Trebisky <t...@mmto.org> wrote:
> Thanks Alex,
>
> I have veered away from ESP8266 temporarily. Now focusing on a little ARM
> based board called the Orange Pi.
> Working up an ethernet driver for it. But I'll be back. ...... I've been
> meaning to participate more on the forum.
> Thanks for the reply.

Are you aware of Corentin Labbe's work on ethernet support for the H3?

https://lwn.net/Articles/717071/

I believe this is getting very close to merged by now.

--
Bye, Peter Korsgaard

[Tom Trebisky]

Thanks for this tip! No I was indeed unaware of this work, and a glance
at it taught me something entirely new,
namely that the H3 emac is a close relative to something else called the
dwmac which may be something from ST
and thus have better documentation. At any event, all this is something
worth looking at.

I work in isolation too much perhaps, one of the benefits of getting
involved in this group.

My driver will be part of the Kyu operating system that I originally
worked up for the Beaglebone Black.

[Paul Sokolovsky]

Hello Tom,

Thanks for this post. I intended to reply long ago, but got caught into
travel and then other real-world stuff which didn't allow me to
prepare a good response. I'm not sure I can prepare as good as I'd like
a reply still, but as it goes out of context, let me try to post at
least something. I apologize in advance if it looks "funky".

On Fri, 3 Mar 2017 14:44:36 -0800 (PST)
Tom Trebisky <t...@mmto.org> wrote:

> On and off over the past year I have been busy disassembling the
> ESP8266 bootrom.
> I am back in the thick of it again lately and have been learning new
> things, and updating my annotated copy on github regularly.
>
> I am aware of Paul's "ScratchAbit", but I started this project over a
> year ago and have gotten familiar enough
> with the xtensa assembly code that I can just keep going like I am.

Right. So, when we communicated at the beginning of January, an example
I wanted to give you where ScratchABit can save you effort is random
undecoded/misdecoded bytes in the middle of instruction stream. You
likely wouldn't get those with ScratchABit. Well, as I mentioned,
holidays suddenly finished, so this argument went unsent. After this
your mail (Mar 3), I wanted to point the same issue again, but then I
noticed that you went thru such cases fixed it!

Ok, my next argument was about the fact that bootrom's .data
and .rodata segments actually get relocated into RAM, and it's helpful
to see that RAM content. To my surprise, you dealt with it too, in
commit
https://github.com/trebisky/esp8266/commit/d74556b3e4dcd6ff4ba1eb0438794c69c373a67c
of Feb 28. Let's see, I had such a code in my local SAB tree at least
since last summer, but all that time I still waited for someone to
pickup ScratchABit to do ESP8266 RE. I gave up by the end of year, and
decided to run such a project myself. I polished and made reusable
the needed code, and setup using it was available since Jan 8 (about a
date we exchanged emails):
https://github.com/pfalcon/xtensa-subjects/commit/6136bf2fad0daca5665ca602f7554ff2480d59fd

So, if you had given SAB a try then, you'd arrive there 1.5 month
earlier.

It goes both ways of course. Soon, very soon (like, circa 2 months of
work) I, with my ScratchABlock decompilation project, will be able to
achieve the same result of cross-referencing MMIO access to functions
as Alex had with his adhoc tools a year or so ago:
http://esp8266-re.foogod.com/wiki/Memory_Accesses_(IoT_RTOS_SDK_0.9.9)

The difference? Alex' is an adhoc tool specific to Xtensa and doing
adhoc, surface analysis of code as available in existing code corpus. I
peered at the entire 130K of its single-module project trying to find
the ground terms of the program analysis like "basic block" or
"reaching definitions", all in vain. Whereas my cute ScratchABlock
thing has nothing to do with Xtensa or any other architecture and is a
generic program transformation framework with already a good inventory
of passes.

> Anyway, I figured I should make others aware of what I am up to, as
> well as to get in touch with this
> group. It may help me or others avoid duplication of effort.

There won't be avoidance of duplication of effort. The whole scene is
built on the duplication of effort. And we should embrace that and
faithfully and thoroughly report our fiascos to each other - because, in
all fairness, I can't call reports like "I've done something which
somebody else did 1.5 months ago" or "I'll do something which somebody
else did a year go" to be "successes", so let me call them "fiascos".

Case by case, month by month, year by year, that may start to ring a
bell. If not to us, then to a new generation of open-source reverse
engineers, who will choose to do it differently. In the meantime though,
everything goes as expected, new people write new adhoc throw-away
tools, e.g. last month's case:
https://github.com/megous/h3-ar100-firmware-decompiler

>
> https://github.com/trebisky/esp8266/tree/master/reverse/bootrom
>
> What most of my effort goes into is in studying the code, and then
> making annotations in the form of comments to document what I have
> learned.

So, any highlights on recent findings? (Mine are of course biased.)

>
> Tom


--
Best regards,
Paul mailto:pmi...@gmail.com