Storable.pm - corrupt when saving to non-truncated file
Brock
In my production environment we have had what we believe to be a
corrupt storable. I am unable to replicate the behaviour in Dev, which
has made it hard to diagnose exactly.
The code has been working for a long time, and the change that made it
break was deleting from the hash. Up until recently, the hash either
stayed the same size, or grew.
The file is opened in readwrite, and then store_fd writes to the file.
As the hash is now (sometimes) smaller, it will write say 1000bytes to
this 2000byte file. The tail 1000 bytes are old, garbage data. In my
test cases, when I retrieve the hash, the garbage data is ignored, as
expected.
open( $sf, "+< $self->{mod_state_filename}" );
flock( $sf, LOCK_EX );
$self->{mod_state} = fd_retrieve($sf);
delete ($self->{mod_state}{"somekey"});
seek( $sf, 0, 0 );
store_fd( $self->{mod_state}, $sf );
flock( $sf, LOCK_UN )
close($sf);
My questions:
1. Should this work, or is it imperative that I truncate the file?
2. Does the stored hash use some kind of file terminator character? If
so, what is it?
3. The above code, deleting and adding and deleting and adding, works
perfectly in my test case. Can you suggest any test case sequence that
might cause it to fail, due to the non-truncated file? (I know this is
a really vague question, so feel free to ignore it).
Thanks, Brock
Nicholas Clark
> The code has been working for a long time, and the change that made it
> break was deleting from the hash. Up until recently, the hash either
> stayed the same size, or grew.
>
> The file is opened in readwrite, and then store_fd writes to the file.
> As the hash is now (sometimes) smaller, it will write say 1000bytes to
> this 2000byte file. The tail 1000 bytes are old, garbage data. In my
> test cases, when I retrieve the hash, the garbage data is ignored, as
> expected.
>
> open( $sf, "+< $self->{mod_state_filename}" );
> flock( $sf, LOCK_EX );
> $self->{mod_state} = fd_retrieve($sf);
> delete ($self->{mod_state}{"somekey"});
> seek( $sf, 0, 0 );
> store_fd( $self->{mod_state}, $sf );
> flock( $sf, LOCK_UN )
> close($sf);
>
> My questions:
> 1. Should this work, or is it imperative that I truncate the file?
Yes, you need to truncate the file.
> 2. Does the stored hash use some kind of file terminator character? If
> so, what is it?
No, it doesn't
> 3. The above code, deleting and adding and deleting and adding, works
> perfectly in my test case. Can you suggest any test case sequence that
> might cause it to fail, due to the non-truncated file? (I know this is
> a really vague question, so feel free to ignore it).
Not easily.
Nicholas Clark
Brock
<snip>
Hi Nicholas,
I've done a bunch more testing, and I can't get it to fail, so for the
moment the delete stays disabled, truncate or not.
Thanks for your answers,
Brock