gob Decoder vs random input
Albert Strasheim
Is the gob Decoder intended to survive random input bytes?
It seems random input causes three kinds of errors in Decode:
- err return
- gob-initiated panic
- runtime panic (invalid argument to makeslice)
I'm willing to contribute some patches if they would be considered.
Regards
Albert
P.S. Example program for Linux:
package main
import (
"fmt"
"bytes"
"gob"
"os"
)
func main() {
fd, err := os.Open("/dev/urandom", os.O_RDONLY, 0)
if err != nil {
panic(err)
}
buf := make([]byte, 8192)
for {
n, err := fd.Read(buf)
if err != nil {
panic(err)
}
dec := gob.NewDecoder(bytes.NewBuffer(buf[:n]))
for {
fmt.Printf("decoding\n")
var e interface{}
err := dec.Decode(&e)
if err != nil {
fmt.Printf("%s\n", err)
continue
}
fmt.Printf("DECODED\n")
}
}
}
Albert Strasheim
On Wed, Jan 19, 2011 at 4:07 PM, Albert Strasheim <ful...@gmail.com> wrote:
> Is the gob Decoder intended to survive random input bytes?
> It seems random input causes three kinds of errors in Decode:
> - err return
> - gob-initiated panic
> - runtime panic (invalid argument to makeslice)
To be more explicit:
We can deal with the error return and recover the gob-initiated panic.
However, runtime panics aren't really something we want to recover
from: we just repanic.
This seems like the most sensible thing to do, since it's hard to
distinguish between a "minor" runtime panic (like gob messing up an
argument to make(...)) and a "major" one (like nil pointer
dereference).
Regards
Albert
Jeff R. Allen
> Is the gob Decoder intended to survive random input bytes?
of gobs as an on-the-wire protocol, but I hadn't thought about the
safety of doing so. (http://blog.nella.org/?p=828)
Seems like there's some work to do before I could safely put my server
on the big bad Internet. :)
Go maintainers, would patches to Gob to to make it bomb-proof be
welcome? Is gob a little experiment/example, or is it intended to be
the canonical serialization format, and thus totally failsafe?
-jeff
Rob 'Commander' Pike
as part of the work, turning some of its panics into errors. it has
too many, and those caused by bad user data should go away.
I worry that people will not check its errors, but let that be on their heads.
-rob
Kai Backman
> I worry that people will not check its errors, but let that be on their heads.
Using gobs as part of the rpc package we have noticed that there are
often panics in the encoders/decoders during routine network flakiness
[1]. I personally much prefer to check errors instead of having to
brace every call to rpc with defer's. Specifically, in my opinion, a
package like gob or rpc should never panic due to malformed input.
Kai
[1] the one we are seeing most is when input is cut short for some reason.
--
Kai Backman, programmer
http://tinkercad.com - The unprofessional solid CAD
Rob 'Commander' Pike
On Jan 21, 2011, at 12:17 PM, Kai Backman wrote:
> On Fri, Jan 21, 2011 at 8:16 PM, Rob 'Commander' Pike <r...@golang.org> wrote:
>> I worry that people will not check its errors, but let that be on their heads.
>
> Using gobs as part of the rpc package we have noticed that there are
> often panics in the encoders/decoders during routine network flakiness
> [1]. I personally much prefer to check errors instead of having to
> brace every call to rpc with defer's. Specifically, in my opinion, a
> package like gob or rpc should never panic due to malformed input.
I think that's what I said.
-rob
Albert Strasheim
On Jan 21, 8:16 pm, "Rob 'Commander' Pike" <r...@golang.org> wrote:
> I'm actively working on the gob code and plan to make it more robust
> as part of the work, turning some of its panics into errors. it has
> too many, and those caused by bad user data should go away.
An attacker or a really unfortunate sequence of corrupted bytes could
cause gob to make a slice that causes a panic due to OOM.
However, it's not obvious how one would distinguish between lots of
valid messages causing OOM and one big invalid message causing OOM.
Maybe a way to specify maximum slice sizes?
If the person calling gob typically puts the received message into a
channel for another goroutine to handle (thinking of some kind of
server here) and you can specify maximum slice sizes, at least you can
more or less compute what your maximum memory footprint is going to be
(channel buffer space * maximum size of messages).
Maybe there's a better idea...
Regards
Albert