AppendOnly file too large

[Edan Maor]

Hi,

I have Redis running using an AOF. I didn't know to periodically run
BGREWRITEAOF, and now, my AOF file has gotten huge (10s of gigas).

Problem is, I can't run BGREWRITEAOF now, because whenever I try to
load the Redis server, it crashes (after taking an hour to try and
load the dataset to memory).

What can I do to fix the AOF file externally? Or fix this problem any
other way?

Thanks!

[Pieter Noordhuis]

There are no currently existing trick I know of that you can apply to
decrease the size. However, I can try and see if I can whip up
something if you can tell me the kinds of commands you use. This does
depend, however, on the type of commands you use. For instance: if you
only use single-key operations (no SORT, SINTER, SUNION, etc), all
operations working against keys that are DEL'ed at some point in the
AOF can be pruned.

Cheers,
Pieter

> --
> You received this message because you are subscribed to the Google Groups "Redis DB" group.
> To post to this group, send email to redi...@googlegroups.com.
> To unsubscribe from this group, send email to redis-db+u...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/redis-db?hl=en.
>
>

[Edan Maor]

I'll get a list together to send.
Before that, I want to make sure my assumptions aren't wrong.
I've copied the file to a larger server, which I hoped would let me
open up Redis and perform the BGREWRITEAOF.

What I'm seeing is the same thing: I load the Redis server, it takes
an hour or so trying to load the dataset into memory, and then it
apparently stops running, since I can no longer connect to it from the
CLI.
I'm now trying to do this once again, and checking Redis's log file to
see what is actually causing it to crash.
Is there anything else I should be doing?