[PATCH 0/3] ramzswap: Eliminate stale data in compressed memory

[Nitin Gupta]

ramzswap driver creates RAM based block devices which can be
used (only) as swap disks. Pages swapped to these disks are
compressed and stored in memory itself.

However, these devices do not get any notification when a swap
slot is freed (swap_map[i] reaches 0). So, we cannot free memory
allocated corresponding to this swap slot. Such stale data can
quickly accumulate in (compressed) memory defeating the whole
purpose of such devices.

To overcome this problem, we now add a callback in 'struct swap_info_struct'
which is called as soon as a swap slot is freed.

Adding handler for this callback:
swapon notifier --> set_swap_free_notify(swap_type, fn)

Removing handler:
swapoff notifier --> set_swap_free_notify(swap_type, NULL)

Alternative approaches:
1) Add callback directly in 'struct block_device_operations' but
that is considered too hacky.
2) Use swap discard mechanism: It involves unncessary overhead of
allocating 'discard bio' requests and its too slow to serve ramzswap
needs.

drivers/staging/ramzswap/ramzswap_drv.c | 91 +++++++++++++++++++++++++++++
drivers/staging/ramzswap/ramzswap_drv.h | 1 +
drivers/staging/ramzswap/ramzswap_ioctl.h | 1 +
include/linux/swap.h | 16 +++++-
mm/swapfile.c | 78 ++++++++++++++++++++++++
5 files changed, 185 insertions(+), 2 deletions(-)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

[Nitin Gupta]

Any chance it can go into 2.6.34? Any comments?

Thanks,
Nitin

[Hugh Dickins]

This is one of the various solutions I disliked already, isn't it?

To me this is just a more convoluted and obscure variant of the
block_device_operations swap_slot_free_notify patch you had in mmotm,
which Linus has rejected.

I admit, I did not understand at all what Linus was proposing with
readpage, writepage and address_space_operations: I kept quiet in
the hope that you'd understand where I didn't!

Hugh

[Nitin Gupta]

On Thu, Mar 11, 2010 at 12:52 PM, Hugh Dickins
<hugh.d...@tiscali.co.uk> wrote:
> On Fri, 5 Mar 2010, Nitin Gupta wrote:

<snip>

>
>> Adding handler for this callback:
>>       swapon notifier --> set_swap_free_notify(swap_type, fn)
>>
>> Removing handler:
>>       swapoff notifier --> set_swap_free_notify(swap_type, NULL)
>>
>>
>> Alternative approaches:
>> 1) Add callback directly in 'struct block_device_operations' but
>> that is considered too hacky.
>> 2) Use swap discard mechanism: It involves unncessary overhead of
>> allocating 'discard bio' requests and its too slow to serve ramzswap
>> needs.
>>
>>  drivers/staging/ramzswap/ramzswap_drv.c   |   91 +++++++++++++++++++++++++++++
>>  drivers/staging/ramzswap/ramzswap_drv.h   |    1 +
>>  drivers/staging/ramzswap/ramzswap_ioctl.h |    1 +
>>  include/linux/swap.h                      |   16 +++++-
>>  mm/swapfile.c                             |   78 ++++++++++++++++++++++++
>>  5 files changed, 185 insertions(+), 2 deletions(-)
>

> This is one of the various solutions I disliked already, isn't it?
>

This is a kind of hybrid of previous approaches. We now have notifier
for swapon/off events only -- which Linus seems to be okay with (earlier
we had notifier for slot free event also). Handler for swapon is responsible
for setting callback for slot free event (earlier we were installing it from
"ramzswap_make_request" directly, when first I/O is received).

> To me this is just a more convoluted and obscure variant of the
> block_device_operations swap_slot_free_notify patch you had in mmotm,
> which Linus has rejected.
>

This slot free notification is a sort of black area for which I have
really run out of
ideas. I tried all the approaches I could think of (and all you and
Pekka suggested)
but nothing seems to satisfy everyone. This is so essential for ramzswap but now
I don't know how to get this done.

> I admit, I did not understand at all what Linus was proposing with
> readpage, writepage and address_space_operations: I kept quiet in
> the hope that you'd understand where I didn't!
>

I guess he is suggesting creating a fictional 'address_space' just like
we have 'struct address_space swapper_space'. This way, we can
intercept writepage naturally in pageout() with new mapping->a_ops->writepage().
And then similarly with mapping->a_ops->readpage().

I'm now looking into this approach. This might allow compressing page cache
pages too. However, figuring out all the details can take significant
amount of time
and allowing ramzswap development to happen in parallel should be okay.

Thanks,
Nitin