anonymous mmap() and random heap allocation

[Xianghua Xiao]

It seems mmap() can not do random allocation for malloc()/heap on
Linux, anyone used pax's 'mmap randonness' option for that? what's the
price for that?

I want to port openbsd's malloc() to linux to avoid heap
crashes(overrun/underrun), openbsd's malloc() can do random allocation
for security reasons, meanwhile it helps to avoid some
overrun/underrun crashes with no extra cost, the latter is what I'm
looking for.

thanks,
xianghua
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

[Mikael Pettersson]

Xianghua Xiao writes:
> It seems mmap() can not do random allocation for malloc()/heap on
> Linux, anyone used pax's 'mmap randonness' option for that? what's the
> price for that?
>
> I want to port openbsd's malloc() to linux to avoid heap
> crashes(overrun/underrun), openbsd's malloc() can do random allocation
> for security reasons, meanwhile it helps to avoid some
> overrun/underrun crashes with no extra cost, the latter is what I'm
> looking for.

You can implement this in user-space with existing kernel features.
1: place guard pages around mmap():ed data
2: parse /proc/self/maps and mmap(MAP_FIXED)
3: use a "safe" compiler or a dynamic binary instrumenter
4: use x86 segments (ugh!)
5: use a safe programming language
..

[Xianghua Xiao]

On Fri, Jul 16, 2010 at 4:59 AM, Mikael Pettersson <mi...@it.uu.se> wrote:
> Xianghua Xiao writes:
>  > It seems mmap() can not do random allocation for malloc()/heap on
>  > Linux, anyone used pax's 'mmap randonness' option for that? what's the
>  > price for that?
>  >
>  > I want to port openbsd's malloc() to linux to avoid heap
>  > crashes(overrun/underrun), openbsd's malloc() can do random allocation
>  > for security reasons, meanwhile it helps to avoid some
>  > overrun/underrun crashes with no extra cost, the latter is what I'm
>  > looking for.
>
> You can implement this in user-space with existing kernel features.
> 1: place guard pages around mmap():ed data
> 2: parse /proc/self/maps and mmap(MAP_FIXED)
> 3: use a "safe" compiler or a dynamic binary instrumenter
> 4: use x86 segments (ugh!)
> 5: use a safe programming language

> ...
>

Mikael,
Thanks.
Forgot to mention this is a powerpc platform. This will be enhancement
to an existing system, i.e. language(mostly C) is already set.
Also, this is for runtime, placing guard pages around mmap() will slow
down the system dramatically.
I'm porting OpenBSD's malloc now. Even though linux's mmap(), unlike
OpenBSD, can not randomize its allocations(which offers free
overrun/underrun protections to some extent), however OpenBSD's malloc
does not put its control block on the heap, that's a plus at least.

What do you mean by 'safe' compiler, something like checkergcc? I
think checkergcc is obsolete now.

cheers,
xianghua