Run strings on a Go binary

[he...@synful.io]

Hi All,

I'm trying to run strings on a Go binary, and it works, but I get a huge amount of output, most of which is junk (I suspect random byte sequences that happen to look like printable characters followed by a null byte). I'm wondering if there's a program that is better suited to Go binaries - are there any properties of Go strings' memory layout that makes them identifiable more easily so I can filter out this junk?

For the curious, this is for a project being developed for a security class in which students are given an intentionally weak software licensing scheme that they must break, and binary analysis is involved to reverse-engineer. At present, it's a pain but doable if you know what you're looking for, but I imagine it will be much more difficult for students who don't know where the vulnerabilities are ahead of time.

Cheers,
synful

[Rob Pike]

http://godoc.org/robpike.io/cmd/strings

-rob

[Joshua Liebow-Feeser]

That's really cool, but it actually gives me more output rather than less because it's aware of a strict superset of strings :)

On Thu, Feb 19, 2015 at 11:00 AM, Rob Pike <r...@golang.org> wrote:

http://godoc.org/robpike.io/cmd/strings

-rob

[thwd]

Use grep to reduce the number of hits via regular expressions. This often proves fruitful.

[Joshua Liebow-Feeser]

I've done that, and it has been very useful, but hasn't gotten me far enough.

On Feb 19, 2015 11:59 AM, "thwd" <sedevel...@gmail.com> wrote:

Use grep to reduce the number of hits via regular expressions. This often proves fruitful.

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/BdWcehcHQEY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[he...@synful.io]

What about this: is there any way to tell the compiler not to do any linking optimizations? I created a toy program that imports all of the same packages in order to get a reference of strings that should be ignored and extracted the strings that only appeared in the target binary (and not the reference one), but I still got a lot of stdlib package strings. I suspect that this is because the compiler is realizing that I'm not using a lot of different functions/types/etc, and optimizing them out of the binary.

On Thursday, February 19, 2015 at 12:01:08 PM UTC-5, Joshua Liebow-Feeser wrote:

I've done that, and it has been very useful, but hasn't gotten me far enough.

On Feb 19, 2015 11:59 AM, "thwd" <sedevel...@gmail.com> wrote:

Use grep to reduce the number of hits via regular expressions. This often proves fruitful.

--
You received this message because you are subscribed to a topic in the Google Groups "golang-nuts" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/golang-nuts/BdWcehcHQEY/unsubscribe.

To unsubscribe from this group and all its topics, send an email to golang-nuts+unsubscribe@googlegroups.com.

[Raffaele Sena]

Also, wouldn't you want to extract the readonly section from the executable and look for strings just in there. There is still more than you may want, but less than what you'll find on the full executable.

objdump will dump just the section (with hex and ascii) but you may need to use some executable parsing library to extra the raw data:

> objdump -s -j .rodata goexecutable

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

[Joshua Liebow-Feeser]

I think strings already does that by default:

$ strings --help
...
 The options are:
  -a - --all                Scan the entire file, not just the data section
...

[Josh Bleecher Snyder]

> I've done that, and it has been very useful, but hasn't gotten me far
> enough.

If what you want is to find all the string literals in a Go program,
they're all in go.string.*:

$ go tool nm helloworld | grep 'go.string.*'
abb30 R go.string.*

Hope that helps,
Josh

[Joshua Liebow-Feeser]

That's hugely helpful; thanks! Given that, do you know how I can figure out where that section ends? My plan is to extract just the bytes in that section and then run strings on those bytes (maybe there's a more intelligent approach?).

[he...@synful.io]

So it looks like these offsets have something funny going on. I used them to extract just the section of the binary that should contain the strings, and didn't find any strings in it other than garbage (including strings I knew to be there). Upon further inspection, I found some string literals in a different part of the binary (not in go.strings.*). In fact, the offset of the strings were before the address of the first section listed by `go tool nm`. This makes the think that perhaps these offsets are not literal byte offsets into the binary, but something else? Perhaps offsets into the address space occupied once the binary is loaded into memory? If this is the case, how would I go about finding the "real" offset (ie, into the binary file)?

[Joshua Liebow-Feeser]

Yep, readelf worked. Doing readelf --sections <binary> prints not only the address of the sections (ie, once they are loaded into memory), but also the offset (into the binary file; the address and offset are different values). Thanks for the tip!

On Fri, Feb 20, 2015 at 8:33 PM, Josh Bleecher Snyder <josh...@gmail.com> wrote:

Drat.

> Perhaps offsets into the address space occupied once the binary is loaded into memory?

Umm...maybe.

> If this is the case, how would I go about finding the "real" offset (ie, into the binary file)?

I don't remember offhand. I'd have to futz around to figure it out.
http://golang.org/pkg/debug/macho/ might help. There's probably a way
to do it with otool or readelf.

This will give you the contents of the relevant symbol:

otool -s __TEXT __rodata helloworld

The addresses on the left appear to correspond to the right range of
bytes for you. So maybe start there?

$ go tool nm -n -size helloworld | grep 'go.string.*'
   abb30     113448 R go.string.*

$ otool -s __TEXT __rodata helloworld | grep -A 5 abb30
00000000000abb30 40 bb 0a 00 00 00 00 00 01 00 00 00 00 00 00 00
00000000000abb40 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00000000000abb50 60 bb 0a 00 00 00 00 00 06 00 00 00 00 00 00 00
00000000000abb60 20 56 61 6c 75 65 00 00 00 00 00 00 00 00 00 00
00000000000abb70 80 bb 0a 00 00 00 00 00 07 00 00 00 00 00 00 00
00000000000abb80 20 56 61 6c 75 65 3e 00 00 00 00 00 00 00 00 00

Note that 20 56 61 6c 75 65 3e is " Value>", which seems promising.

Sorry for not having a pat answer offhand. Please post back to the list
if you figure it out.

-josh

[Joshua Liebow-Feeser]

If anyone is curious what their handiwork has made possible, here's the final (now released) assignment: http://cs.brown.edu/courses/cs1951e/files/free-trial.pdf

[Rick Tait]

On Sunday, February 22, 2015 at 1:34:46 PM UTC-8, Joshua Liebow-Feeser wrote:

If anyone is curious what their handiwork has made possible, here's the final (now released) assignment: http://cs.brown.edu/courses/cs1951e/files/free-trial.pdf

Joshua -- that was a really, really cool assignment. i wish i'd have received such assignments when i was at University, and that my profs were as switched on as they seem to be these days! 

Also, loved that a README is 40% of a grade on something. ;) 

thanks for posting your followup with the assignment. good stuff indeed!

-RMT

[Joshua Liebow-Feeser]

Glad you like it! I'll let you know how it goes :)

--

[Dan Kortschak]

Will the binary be made publicly available after the course? That sounds like fun.

You received this message because you are subscribed to the Google Groups "golang-nuts" group.

To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.