Recompiling and replacing a function at runtime?

[Peret Finctor]

Sometimes I find myself in need to edit one of the functions I have, and restarting the whole program just for that is kind of... Let's say I'm too lazy for that. It seems that recompiling and replacing (hot patching? monkey patching?) the whole function at runtime is the simpliest way to edit code at runtime. Well, using interpreted language like python would be even simplier, but I don't want to depend on unnecessary code forever just because it's easier to code with it.

I found how to do it in C, and found that Microsoft were doing something like this since forever. http://www.codeproject.com/Articles/27339/API-hooking-for-hotpatchable-operating-systems . I wonder what kind of problems may appear if I'll try to do something simular with a smarter compiler, like Go.

Um, and, while I did found how it can be possibly done, I haven't done that yet. Have you met something simular somewhere? This technique should be much more popular than it currently is.

[Giulio Iotti]

On Wednesday, November 11, 2015 at 10:45:54 AM UTC+1, Peret Finctor wrote:

Sometimes I find myself in need to edit one of the functions I have, and restarting the whole program just for that is kind of... Let's say I'm too lazy for that. It seems that recompiling and replacing (hot patching? monkey patching?) the whole function at runtime is the simpliest way to edit code at runtime. Well, using interpreted language like python would be even simplier, but I don't want to depend on unnecessary code forever just because it's easier to code with it.

Have you looked at Gin[1] and friends?

Of course the approach is different. A self modifying softwar--the way you describe it--is a perfect recipe for malware.

[1] https://github.com/codegangsta/gin

-- 

Giulio Iotti

[Dan Kortschak]

Don't do it, but http://bouk.co/blog/monkey-patching-in-go/.

[Peret Finctor]

Um, this solves only half of a problem. Described monkey-patching only works if both replacing and replaced functions already compiled to binary form and already in memory. But I need to also recompile the function from the source code to binary, and then replace it, without restarting program. This means that all addresses (pointers?) used in the replacing function should be exactly the same as in replaced one. Something like dinamic linking, runtime linking, what's the word for it.

[Peret Finctor]

Have you looked at Gin[1] and friends?

Huh, this just recompiles and reruns program through the command line? That's an interesting way too, why I didn't thought about it...
This solution isn't as flexible as I want. I want to be able to edit anything at any time.
 

 

Of course the approach is different. A self modifying softwar--the way you describe it--is a perfect recipe for malware.

You think? You know, unlike developing in a scripting language (where you will forever depend on it's runtime untill you'll decide to completely rewrite your program in something like C), in this approach the only difference between the "quick development mode" and the "ready to release mode" is a seven bytes of trash before every function. This should never affect users in any way, if they are getting the version without the leading space.

Malvare thing is more about the protection of OS itself. This link https://github.com/bouk/monkey mentions, that "Monkey won't work on some security-oriented operating system that don't allow memory pages to be both write and execute at the same time". I'd like to know more about which OSes are in that list.

[Giulio Iotti]

On Thursday, November 12, 2015 at 7:25:11 AM UTC+1, Peret Finctor wrote:

Um, this solves only half of a problem. Described monkey-patching only works if both replacing and replaced functions already compiled to binary form and already in memory. But I need to also recompile the function from the source code to binary, and then replace it, without restarting program. This means that all addresses (pointers?) used in the replacing function should be exactly the same as in replaced one. Something like dinamic linking, runtime linking, what's the word for it.

Dynamic linking is the word, but I understand it works differently: symbols are resolved at startup time, not when you call/use a symbol itself.

What you want is something based on dlopen/dlsym[1]. I am not sure what the status of these is in Go, but I suspect they are not ready yet as dynamic linking is something new in 1.5.

In any case we have someone who knows "something" (I love understatements) about dynamic linking. Ian if you read us, say hi.

[1] man 3 dlopen -- notice it's a libc thing, not a syscall.

-- 

Giulio Iotti

[Giulio Iotti]

On Thursday, November 12, 2015 at 7:47:32 AM UTC+1, Peret Finctor wrote:

Of course the approach is different. A self modifying softwar--the way you describe it--is a perfect recipe for malware.

You think? You know, unlike developing in a scripting language (where you will forever depend on it's runtime untill you'll decide to completely rewrite your program in something like C), in this approach the only difference between the "quick development mode" and the "ready to release mode" is a seven bytes of trash before every function. This should never affect users in any way, if they are getting the version without the leading space.

I don't get the seven bytes of trash reference. Stack protection?

 

Malvare thing is more about the protection of OS itself. This link https://github.com/bouk/monkey mentions, that "Monkey won't work on some security-oriented operating system that don't allow memory pages to be both write and execute at the same time". I'd like to know more about which OSes are in that list.

 This monkey package calls syscall.Mproctect exactly to address the problem I wrote before. There are good reasons why a segment should be either executable or writable... I assume know about "stack smashing" and the like (which thankfully are still harder to achieve in Go even with writable/executable segment.) 

-- 

Giulio Iotti

[Peret Finctor]

четверг, 12 ноября 2015 г., 11:03:46 UTC+4 пользователь Giulio Iotti написал

Dynamic linking is the word, but I understand it works differently: symbols are resolved at startup time, not when you call/use a symbol itself.

What you want is something based on dlopen/dlsym[1]. I am not sure what the status of these is in Go, but I suspect they are not ready yet as dynamic linking is something new in 1.5.

In any case we have someone who knows "something" (I love understatements) about dynamic linking. Ian if you read us, say hi.

[1] man 3 dlopen -- notice it's a libc thing, not a syscall.

-- 

Giulio Iotti

If I'll use dlopen/dlsym for that I want, then I'll have to make every single function I have as a .dll (.so?). That would be problematic.

[Giulio Iotti]

On Thursday, November 12, 2015 at 8:15:05 AM UTC+1, Peret Finctor wrote:

If I'll use dlopen/dlsym for that I want, then I'll have to make every single function I have as a .dll (.so?). That would be problematic.

Oh yes, it would.

But why do you want to do this? Why isn't a "graceful restart" of a daemon not enough for you?

I'm afraid I didn't really get your aim ("I have problem X, this is the solution I am thinking of...")

-- 

Giulio Iotti

[Peret Finctor]

четверг, 12 ноября 2015 г., 11:07:53 UTC+4 пользователь Giulio Iotti написал:

On Thursday, November 12, 2015 at 7:47:32 AM UTC+1, Peret Finctor wrote:

Of course the approach is different. A self modifying softwar--the way you describe it--is a perfect recipe for malware.

You think? You know, unlike developing in a scripting language (where you will forever depend on it's runtime untill you'll decide to completely rewrite your program in something like C), in this approach the only difference between the "quick development mode" and the "ready to release mode" is a seven bytes of trash before every function. This should never affect users in any way, if they are getting the version without the leading space.

I don't get the seven bytes of trash reference. Stack protection?

I was referencing http://www.codeproject.com/Articles/27339/API-hooking-for-hotpatchable-operating-systems

In short, microsoft  added a bit of trash to their functions: 5 nop operands that do nothing, placeholder for a jmp instruction to jump to a new function, and a "push ebp; mov ebp,esp" combo that also does nothing, placeholder for a jmp to those 5 nop above. This trick allows to redirect code execution to whenever we want, for any function where this placeholder is placed.

 

Malvare thing is more about the protection of OS itself. This link https://github.com/bouk/monkey mentions, that "Monkey won't work on some security-oriented operating system that don't allow memory pages to be both write and execute at the same time". I'd like to know more about which OSes are in that list.

 This monkey package calls syscall.Mproctect exactly to address the problem I wrote before. There are good reasons why a segment should be either executable or writable... I assume know about "stack smashing" and the like (which thankfully are still harder to achieve in Go even with writable/executable segment.) 

Probably he means that on some OSes this syscall.Mproctect (or VirtualProtect in winapi terms, I guess it's the same thing) doesn't work. Probably.

-- 

Giulio Iotti

четверг, 12 ноября 2015 г., 11:19:35 UTC+4 пользователь Giulio Iotti написал:

That's not really a problem, more like I'm "striving to be lazy". It will also have a set of advantages, like probably speeding me up like 10 times, freeing me from writing config files (I'll be able to edit code and variables directly), blurring the border between code and data, scripting and compiling languages, using C (or Go) in the way people are using lisp, making more readable emacs, etc...
 

 

-- 

Giulio Iotti

[Giulio Iotti]

On Thursday, November 12, 2015 at 8:36:54 AM UTC+1, Peret Finctor wrote:

четверг, 12 ноября 2015 г., 11:07:53 UTC+4 пользователь Giulio Iotti написал:

On Thursday, November 12, 2015 at 7:47:32 AM UTC+1, Peret Finctor wrote:

Of course the approach is different. A self modifying softwar--the way you describe it--is a perfect recipe for malware.

You think? You know, unlike developing in a scripting language (where you will forever depend on it's runtime untill you'll decide to completely rewrite your program in something like C), in this approach the only difference between the "quick development mode" and the "ready to release mode" is a seven bytes of trash before every function. This should never affect users in any way, if they are getting the version without the leading space.

I don't get the seven bytes of trash reference. Stack protection?

I was referencing http://www.codeproject.com/Articles/27339/API-hooking-for-hotpatchable-operating-systems

In short, microsoft  added a bit of trash to their functions: 5 nop operands that do nothing, placeholder for a jmp instruction to jump to a new function, and a "push ebp; mov ebp,esp" combo that also does nothing, placeholder for a jmp to those 5 nop above. This trick allows to redirect code execution to whenever we want, for any function where this placeholder is placed.

Didn't know this, interesting. Thank you!

 

 This monkey package calls syscall.Mproctect exactly to address the problem I wrote before. There are good reasons why a segment should be either executable or writable... I assume know about "stack smashing" and the like (which thankfully are still harder to achieve in Go even with writable/executable segment.) 

Probably he means that on some OSes this syscall.Mproctect (or VirtualProtect in winapi terms, I guess it's the same thing) doesn't work. Probably.

Most probably, makes sense. I have no idea what the status of this is... For Linux, probably it's up to stuff like Apparmor to enforce such things.

 

That's not really a problem, more like I'm "striving to be lazy". It will also have a set of advantages, like probably speeding me up like 10 times, freeing me from writing config files (I'll be able to edit code and variables directly), blurring the border between code and data, scripting and compiling languages, using C (or Go) in the way people are using lisp, making more readable emacs, etc...

Oh, a hack. Let's get wild ideas then ;)

-- 

Giulio Iotti 

[Benjamin Measures]

On Thursday, 12 November 2015 07:36:54 UTC, Peret Finctor wrote:

This trick allows to redirect code execution to whenever we want, for any function where this placeholder is placed.

Sounds perfect for what an attacker might need.

https://cansecwest.com/slides/2013/DEP-ASLR%20bypass%20without%20ROP-JIT.pdf

> Really like a GIFT from an unknown coder

[Peret Finctor]

Didn't really understood what's this presentation is about. Also please read my post here https://groups.google.com/d/msg/golang-nuts/39IX6vA2VsA/7aQV1avBDwAJ

четверг, 12 ноября 2015 г., 21:47:14 UTC+4 пользователь Benjamin Measures написал:

[Benjamin Measures]

That doesn't give licence for compilers to avoid all responsibility and not forgo implementing security features such as stack-smashing protection and position-independent executabes. Similarly, any feature introduced by a compiler that provides a "gift" to attackers must consider the implications responsibly.

[Benjamin Measures]

On Friday, 13 November 2015 10:04:16 UTC, Benjamin Measures wrote:

That doesn't give licence for compilers to avoid all responsibility and not forgo implementing security features such as stack-smashing protection and position-independent executabes. Similarly, any feature introduced by a compiler that provides a "gift" to attackers must consider the implications responsibly.

s/and not forgo implementing/and forgo implementing/

[Giulio Iotti]

On Friday, November 13, 2015 at 11:04:16 AM UTC+1, Benjamin Measures wrote:

That doesn't give licence for compilers to avoid all responsibility and not forgo implementing security features such as stack-smashing protection and position-independent executabes. Similarly, any feature introduced by a compiler that provides a "gift" to attackers must consider the implications responsibly.

Honest question: how does PIE help, security-wise? Address space layout randomization?

-- 

Giulio Iotti

[Benjamin Measures]

On Friday, 13 November 2015 10:11:37 UTC, Giulio Iotti wrote:

Honest question: how does PIE help, security-wise? Address space layout randomization?

Yes, it allows ASLR to be extended to the executable (ie. not just dynamically linked code).