RFC: direct MTD support for SquashFS
Ferenc Wagner
In embedded systems, SquashFS over MTD would be a considerable win, as
that would permit configuring without CONFIG_BLOCK. Please find
attached a naive patch against 2.6.33 for this. It does not handle bad
MTD blocks, that could be handled by gluebi (once you're willing to take
the UBI overhead), or by a custom solution later.
For now, 2.6.34 gained pluggable decompressors, so this patch does not
apply anymore, though the main idea holds. My questions: is the
community interested in integrating something like this, should this
patch transformed into something acceptable, or am I a total lunatic?
I don't know a thing about filesystem development, but willing to learn
and refactor. Comments welcome.
--
Thanks,
Feri.
Peter Korsgaard
Ferenc> Hi,
Ferenc> In embedded systems, SquashFS over MTD would be a considerable
Ferenc> win, as that would permit configuring without CONFIG_BLOCK.
Ferenc> Please find attached a naive patch against 2.6.33 for this. It
Ferenc> does not handle bad MTD blocks, that could be handled by gluebi
Ferenc> (once you're willing to take the UBI overhead), or by a custom
Ferenc> solution later.
Ferenc> For now, 2.6.34 gained pluggable decompressors, so this patch
Ferenc> does not apply anymore, though the main idea holds. My
Ferenc> questions: is the community interested in integrating something
Ferenc> like this, should this patch transformed into something
Ferenc> acceptable, or am I a total lunatic? I don't know a thing
Ferenc> about filesystem development, but willing to learn and
Ferenc> refactor. Comments welcome.
Nice, I have been thinking about that as well. What kind of size savings
are you getting with this?
CC'ing linux-embedded as this might be of interest there as well.
--
Bye, Peter Korsgaard
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Vitaly Wool
>>>>>> "Ferenc" == Ferenc Wagner <wf...@niif.hu> writes:
> Ferenc> For now, 2.6.34 gained pluggable decompressors, so this patch
> Ferenc> does not apply anymore, though the main idea holds. My
> Ferenc> questions: is the community interested in integrating something
> Ferenc> like this, should this patch transformed into something
> Ferenc> acceptable, or am I a total lunatic? I don't know a thing
> Ferenc> about filesystem development, but willing to learn and
> Ferenc> refactor. Comments welcome.
>
> Nice, I have been thinking about that as well. What kind of size savings
> are you getting with this?
Yeah, I'm interested in that as well.
~Vitaly
Ferenc Wagner
>>>>>> "Ferenc" == Ferenc Wagner <wf...@niif.hu> writes:
>
> Ferenc> In embedded systems, SquashFS over MTD would be a considerable
> Ferenc> win, as that would permit configuring without CONFIG_BLOCK.
> Ferenc> Please find attached a naive patch against 2.6.33 for this. It
> Ferenc> does not handle bad MTD blocks, that could be handled by gluebi
> Ferenc> (once you're willing to take the UBI overhead), or by a custom
> Ferenc> solution later.
>
> Ferenc> For now, 2.6.34 gained pluggable decompressors, so this patch
> Ferenc> does not apply anymore, though the main idea holds. My
> Ferenc> questions: is the community interested in integrating something
> Ferenc> like this, should this patch transformed into something
> Ferenc> acceptable, or am I a total lunatic? I don't know a thing
> Ferenc> about filesystem development, but willing to learn and
> Ferenc> refactor. Comments welcome.
>
> Nice, I have been thinking about that as well. What kind of size savings
> are you getting with this?
I could only compare apples to oranges before porting the patch to the
LZMA variant. So I refrain from that for a couple of days yet. But
meanwhile I started adding a pluggable backend framework to SquashFS,
and would much appreciate some comments about the applicability of this
idea. The patch is (intended to be) a no-op, applies on top of current
git (a3d3203e4bb40f253b1541e310dc0f9305be7c84).
--
Thanks,
Feri.
Phillip Lougher
>
> I could only compare apples to oranges before porting the patch to the
> LZMA variant. So I refrain from that for a couple of days yet. But
> meanwhile I started adding a pluggable backend framework to SquashFS,
> and would much appreciate some comments about the applicability of this
> idea. The patch is (intended to be) a no-op, applies on top of current
> git (a3d3203e4bb40f253b1541e310dc0f9305be7c84).
This looks promising, making the backend pluggable (like the new
compressor framework) is far better and cleaner than scattering the
code full of #ifdef's. Far better than the previous patch :-)
A couple of specific comments...
+/* A backend is initialized for each SquashFS block read operation,
+ * making further sequential reads possible from the block.
+ */
+static void *bdev_init(struct squashfs_sb_info *msblk, u64 index,
size_t length)
+{
+ struct squashfs_bdev *bdev = msblk->backend_data;
+ struct buffer_head *bh;
+
+ bh = kcalloc((msblk->block_size >> bdev->devblksize_log2) + 1,
+ sizeof(*bh), GFP_KERNEL);
You should alloc against the larger of msblk->block_size and
METADATA_SIZE (8 Kbytes). Block_size could be 4 Kbytes only.
+static int fill_bdev_super(struct super_block *sb, void *data, int silent)
+{
+ struct squashfs_sb_info *msblk;
+ struct squashfs_bdev *bdev;
+ int err = squashfs_fill_super2(sb, data, silent, &squashfs_bdev_ops);
+ if (err)
+ return err;
+
+ bdev = kzalloc(sizeof(*bdev), GFP_KERNEL);
+ if (!bdev)
+ return -ENOMEM;
+
+ bdev->devblksize = sb_min_blocksize(sb, BLOCK_SIZE);
+ bdev->devblksize_log2 = ffz(~bdev->devblksize);
+
+ msblk = sb->s_fs_info;
+ msblk->backend_data = bdev;
+ return 0;
+}
This function looks rather 'back-to-front' to me. I'm assuming that
squashfs_fill_super2() will be the current fill superblock function?
This function wants to read data off the filesystem through the
backend, and yet the backend (bdev, mblk->backend_data) hasn't been
initialised when it's called...
Phillip
Ferenc Wagner
> On Thu, Mar 18, 2010 at 4:38 PM, Ferenc Wagner <wf...@niif.hu> wrote:
>
>> I could only compare apples to oranges before porting the patch to the
>> LZMA variant. So I refrain from that for a couple of days yet. But
>> meanwhile I started adding a pluggable backend framework to SquashFS,
>> and would much appreciate some comments about the applicability of this
>> idea. The patch is (intended to be) a no-op, applies on top of current
>> git (a3d3203e4bb40f253b1541e310dc0f9305be7c84).
>
> This looks promising, making the backend pluggable (like the new
> compressor framework) is far better and cleaner than scattering the
> code full of #ifdef's. Far better than the previous patch :-)
Yeah, the previous patch was only a little bit more than a proof that I
can make SquashFS work on an MTD device. The MTD access part is
probably the only thing to criticize there: maybe it would be better
done in blocks of some particular size, via a different interface.
> +static void *bdev_init(struct squashfs_sb_info *msblk, u64 index,
> size_t length)
> +{
> + struct squashfs_bdev *bdev = msblk->backend_data;
> + struct buffer_head *bh;
> +
> + bh = kcalloc((msblk->block_size >> bdev->devblksize_log2) + 1,
> + sizeof(*bh), GFP_KERNEL);
>
> You should alloc against the larger of msblk->block_size and
> METADATA_SIZE (8 Kbytes). Block_size could be 4 Kbytes only.
Hmm, okay. Though this code is a verbatim copy of that in block.c.
> +static int fill_bdev_super(struct super_block *sb, void *data, int silent)
> +{
> + struct squashfs_sb_info *msblk;
> + struct squashfs_bdev *bdev;
> + int err = squashfs_fill_super2(sb, data, silent, &squashfs_bdev_ops);
> + if (err)
> + return err;
> +
> + bdev = kzalloc(sizeof(*bdev), GFP_KERNEL);
> + if (!bdev)
> + return -ENOMEM;
> +
> + bdev->devblksize = sb_min_blocksize(sb, BLOCK_SIZE);
> + bdev->devblksize_log2 = ffz(~bdev->devblksize);
> +
> + msblk = sb->s_fs_info;
> + msblk->backend_data = bdev;
> + return 0;
> +}
>
> This function looks rather 'back-to-front' to me. I'm assuming that
> squashfs_fill_super2() will be the current fill superblock function?
Yes, with the extra parameter added.
> This function wants to read data off the filesystem through the
> backend, and yet the backend (bdev, mblk->backend_data) hasn't been
> initialised when it's called...
It can't be, because msblk = sb->s_fs_info is allocated by
squashfs_fill_super(). Now it will be passed the ops, so after
allocating msblk it can also fill out the ops. After that it can read,
and squashfs_read_data() will call the init, read and free operations of
the backend. The backend itself has no persistent state between calls
to squashfs_read_data(). Btw. struct super_block has fields named
s_blocksize and s_blocksize_bits, aren't those the same as devblksize
and devblksize_log in squashfs_sb_info? (They are being moved into
backend_data by the above.) If yes, shouldn't they be used instead?
While we're at it: is it really worth submitting all the buffer heads
at the beginning, instead of submitting them one at a time as needed by
the decompression process and letting the IO scheduler do readahead and
request coalescing as it sees fit? At the very least, that would
require less memory, while possibly not hurting performance too much.
On the other hand, would it be possible to avoid the memory copy of
uncompressed blocks by doing a straight (DMA) transfer from the device
into the page cache?
LZMA support is not in mainline yet, but I saw that unlzma is done in a
single step, which requires block-sized input and output buffers. Is
there any particular reason it's done this way, not chunk-by-chunk as
inflate? This easily costs hundreds of kilobytes of virtual memory,
which isn't negligible on embedded systems.
--
Thanks for your comments,
Feri.
Ferenc Wagner
> Phillip Lougher <phillip...@gmail.com> writes:
>
>> On Thu, Mar 18, 2010 at 4:38 PM, Ferenc Wagner <wf...@niif.hu> wrote:
>>
>> +static int fill_bdev_super(struct super_block *sb, void *data, int silent)
>> +{
>> + struct squashfs_sb_info *msblk;
>> + struct squashfs_bdev *bdev;
>> + int err = squashfs_fill_super2(sb, data, silent, &squashfs_bdev_ops);
>> + if (err)
>> + return err;
>> +
>> + bdev = kzalloc(sizeof(*bdev), GFP_KERNEL);
>> + if (!bdev)
>> + return -ENOMEM;
>> +
>> + bdev->devblksize = sb_min_blocksize(sb, BLOCK_SIZE);
>> + bdev->devblksize_log2 = ffz(~bdev->devblksize);
>> +
>> + msblk = sb->s_fs_info;
>> + msblk->backend_data = bdev;
>> + return 0;
>> +}
>>
>> This function looks rather 'back-to-front' to me. I'm assuming that
>> squashfs_fill_super2() will be the current fill superblock function?
>
> Yes, with the extra parameter added.
>
>> This function wants to read data off the filesystem through the
>> backend, and yet the backend (bdev, mblk->backend_data) hasn't been
>> initialised when it's called...
>
> It can't be, because msblk = sb->s_fs_info is allocated by
> squashfs_fill_super(). Now it will be passed the ops, so after
> allocating msblk it can also fill out the ops. After that it can read,
> and squashfs_read_data() will call the init, read and free operations of
> the backend.
And here we indeed have a rather fundamental problem. This isn't
specific to the discussed plugin system at all. Even in the current
code, to set msblk->block_size squashfs_fill_super() calls
squashfs_read_table() to read the superblock, which in turn calls
squashfs_read_data(), which uses msblk->block_size to allocate enough
buffer heads, but msblk->block_size just can't be set at this point.
msblk->bytes_used is preset with a dummy value to make the read
possible, but msblk->block_size is not. Fortunately, one buffer head is
allocated each time nevertheless. I wonder what a correct solution
would look lke..
--
Regards,
Feri.
Phillip Lougher
Block_size is known to be zero (the structure has been zeroed out at
alloc), and so it is known that the one block alloced in this case
will be correct.
Congratulations you've managed to really piss me off in your third or so email.
Cheers
Phillip
Ferenc Wagner
>> would look like..
>
> Block_size is known to be zero (the structure has been zeroed out at
> alloc), and so it is known that the one block alloced in this case
> will be correct.
If block_size=0 is always a good dummy value for this single call,
that's great. Fixing this in a general way in the backend framework
might require allocating and partly initializing squashfs_sb_info in the
backend specific fill_super() function, before calling squashfs_fill_super()
for finalizing it. Even though it may be possible to work around this
for the bdev or mtd backends, it probably isn't worth it.
> Congratulations you've managed to really piss me off in your third or
> so email.
Sorry, I'm not sure I understand. If you mean that I made an ass of
myself by my questions, that's OK, I'm certainly a newbie and I
admittedly have no idea what I'm fiddling with. Please feel free to
ignore stupid questions. On the other hand, if you mean that I hurt
your feelings in any way, I'd like to apologize: it certainly wasn't my
intention, but I might have chosen inappropriate terms. Sorry for that.
--
Cheers,
Ferenc Wagner
> A couple of specific comments...
>
> +/* A backend is initialized for each SquashFS block read operation,
> + * making further sequential reads possible from the block.
> + */
> +static void *bdev_init(struct squashfs_sb_info *msblk, u64 index,
> size_t length)
> +{
> + struct squashfs_bdev *bdev = msblk->backend_data;
> + struct buffer_head *bh;
> +
> + bh = kcalloc((msblk->block_size >> bdev->devblksize_log2) + 1,
> + sizeof(*bh), GFP_KERNEL);
>
> You should alloc against the larger of msblk->block_size and
> METADATA_SIZE (8 Kbytes). Block_size could be 4 Kbytes only.
I plugged in a max(). Couldn't that trailing +1 be converted into a +2
like this?
bh = kcalloc((max(msblk->block_size, METADATA_SIZE) + 2) >> bdev->devblksize_log2
> +static int fill_bdev_super(struct super_block *sb, void *data, int silent)
>
> This function looks rather 'back-to-front' to me. I'm assuming that
> squashfs_fill_super2() will be the current fill superblock function?
> This function wants to read data off the filesystem through the
> backend, and yet the backend (bdev, mblk->backend_data) hasn't been
> initialised when it's called...
I solved it by introducing a callback function for adding the backend.
That may be overkill, but it seems to give the most shared code.
The attached patch series survived some testing here. My only doubt:
the current backend interface necessitates a memory copy from the buffer
heads. This is no problem for mtd and lzma which copy the data anyway,
but makes this code less efficient in the bdev+zlib case.
I've got one more patch, which I forgot to export, to pull out the
common logic from the backend init functions back into squashfs_read_data().
With the bdev backend, that entails reading the first block twice in a
row most of the time. This again could be worked around by extending
the backend interface, but I'm not sure if it's worth it.
How does this look like now?
--
Regards,
Feri.
Ferenc Wagner
> I've got one more patch, which I forgot to export, to pull out the
> common logic from the backend init functions back into squashfs_read_data().
> With the bdev backend, that entails reading the first block twice in a
> row most of the time. This again could be worked around by extending
> the backend interface, but I'm not sure if it's worth it.
Here it is. I also corrected the name of SQUASHFS_METADATA_SIZE, so it
may as well compile now.
--
Regards,
Feri.
Phillip Lougher
Thanks for your patches.
First things first. Patch sending etiquette :-)
1. Please don't send separate git commits in one big email, it makes them
hard to cross-reference (I lost count of the times I had to repeatedly
scroll though the email to check an earlier commit).
2. Please don't send a patch series consisting of your development process.
Reviewing takes effort, having reviewed one commit to find it reworked
in a second commit, and then reworked again, as the code evolves from
a -> b -> c-> d is irritating, it makes the final solution harder to
evaluate (because you have to keep all the changes in your head), and
wastes my time.
3. Incremental patches are valid if they break up a patch series into easily
digestible changes which can be separately understood, but not if they're
just reworking your code as development progresses...
OK, now for the specific comments.
Frameworks are supposed to make the code cleaner, more understandable,
and generally better. Unfortunately, I see no evidence of that in your
patch. Sorry to be blunt, but there it is.
The backend has seven functions!
+ void *(*init)(struct squashfs_sb_info *, u64, size_t);
+ void (*free)(struct squashfs_sb_info *);
+ ssize_t (*read)(struct squashfs_sb_info *, void **, size_t);
+ int (*probe)(struct file_system_type *, int, const char *,
+ void*, struct vfsmount *);
+ void (*kill)(struct squashfs_sb_info *);
+ loff_t (*size)(const struct super_block *);
+ const char *(*devname)(const struct super_block *, char *);
We move from a single read() function to requiring seven functions to do
the same work, just to add mtd support...
Reading the superblock changes into a 6 function deep nightmare
squashfs_find_backend -> probe -> get_sb_dev -> fill_bdev_super ->
squashfs_fill_super -> add_backend
I find the current 3 function deep situation forced by VFS already a mess,
squashfs_get_sb -> get_sb_bdev -> squashfs_fill_super
Reading a block now takes 3 function calls, init, read, and free. Plus in
your last commit you make the huge mistake of preferring code elegance over
performance, and resort to calling init, read, and free *twice*, the first
just to read *two* bytes (yes, 2 bytes). Why do you think the code was
coded that way in the first place? A fit of absent mindedness perhaps? No, for
performance reasons.
Secondly you make the classic mistake of concentrating on what you want to do
(add MTD), and not giving a damn about anything else. You've totally broken all
the read optimisations in zlib decompression for block devices. Buffer_heads
are deliberately passed directly to the zlib decompressor to allow it to
optimise performance (pass the buffer_head directly to zlib etc.). Buffer_heads
are exposed to the decompressors without crappy go-between wrappers deliberately.
Unfortunately there are lots of other instances where you've eliminated carefully
optimised code.
That's another of my major issues, the patch seems hugely gratuitous, it
touches a huge amount of files/code it shouldn't do, much of this slicing up
optimisied heavily tested and fragile code into other files/functions hither
and thither. Unfortunately much of this code took a long time to get
right, and suffered numerous unanticipated edge conditions/fsfuzzer
triggered bugs. I only touch this code with caution and nothing like to
the extent you've subjected it to.
In short this doesn't pass my quality threshold or the make the minimum changes
necessary threshold. I wouldn't consider asking Linus to merge this.
BTW as a comparison, I have added MTD support to Squashfs twice in the
past, *with* bad block support. The latest has a diff patch of ~500 lines,
with far less complexity and code changes necessary.
BTW2 as evidenced by your FIXME comments against my code in your patch,
you really have to get over the notion that if you don't understand it,
the code must be wrong.
Cheers
Peter Korsgaard
Hi,
Phillip> BTW as a comparison, I have added MTD support to Squashfs
Phillip> twice in the past, *with* bad block support. The latest has a
Phillip> diff patch of ~500 lines, with far less complexity and code
Phillip> changes necessary.
Could we merge that then please?
Again, what kind of size reduction can we expect from getting rid of the
block layer?
--
Bye, Peter Korsgaard
Ferenc Wagner
>>>>>> "Phillip" == Phillip Lougher <phi...@lougher.demon.co.uk> writes:
>
> Phillip> BTW as a comparison, I have added MTD support to Squashfs
> Phillip> twice in the past, *with* bad block support. The latest has a
> Phillip> diff patch of ~500 lines, with far less complexity and code
> Phillip> changes necessary.
>
> Again, what kind of size reduction can we expect from getting rid of
> the block layer?
My size-optimized OpenWRT vmlinux shrank ~170 kB after disabling
CONFIG_BLOCK. Similarly for the memory consumption.
--
Regards,
Feri.
Peter Korsgaard
Hi,
>> Again, what kind of size reduction can we expect from getting rid of
>> the block layer?
Ferenc> My size-optimized OpenWRT vmlinux shrank ~170 kB after disabling
Ferenc> CONFIG_BLOCK. Similarly for the memory consumption.
Neat, thanks - That sounds worthwhile.
--
Bye, Peter Korsgaard
Ferenc Wagner
> Ferenc Wagner wrote:
>
>> Now with the patch series, sorry.
>>
>> Ferenc Wagner <wf...@niif.hu> writes:
>>
>>> I've got one more patch, which I forgot to export, to pull out the
>>> common logic from the backend init functions back into squashfs_read_data().
>>> With the bdev backend, that entails reading the first block twice in a
>>> row most of the time. This again could be worked around by extending
>>> the backend interface, but I'm not sure if it's worth it.
>>
>> Here it is. I also corrected the name of SQUASHFS_METADATA_SIZE, so it
>> may as well compile now.
>
> Thanks for your patches.
>
> First things first. Patch sending etiquette :-) [...]
Points taken. About the splits, they were the result of quite some
rebasing: I tried to give each patch a unique focus, but apparently
failed. Only the last one touches substantial changed code, for the
explicitly stated reason: it hurts performance.
> Frameworks are supposed to make the code cleaner, more understandable,
> and generally better.
This framework is about added flexibility. I agree it requires some
added complexity, but I hope we can manage it.
> Unfortunately, I see no evidence of that in your patch. Sorry to be
> blunt, but there it is.
No problem.
> + void *(*init)(struct squashfs_sb_info *, u64, size_t);
> + void (*free)(struct squashfs_sb_info *);
> + ssize_t (*read)(struct squashfs_sb_info *, void **, size_t);
> + int (*probe)(struct file_system_type *, int, const char *,
> + void*, struct vfsmount *);
> + void (*kill)(struct squashfs_sb_info *);
> + loff_t (*size)(const struct super_block *);
> + const char *(*devname)(const struct super_block *, char *);
>
> We move from a single read() function to requiring seven functions to do
> the same work, just to add mtd support...
Your single read() function also had separate init, read and free
phases, with something generic in between. I can try to invert the
structure, but see below.
> Reading the superblock changes into a 6 function deep nightmare
> squashfs_find_backend -> probe -> get_sb_dev -> fill_bdev_super ->
> squashfs_fill_super -> add_backend
It's convoluted, but only moves original code around. If you prefer, I
can simplify that by embedding squashfs_sb_info into backend-specific
structures and allocating those separately. This would also get rid of
some of the above 7 functions.
> Reading a block now takes 3 function calls, init, read, and free.
Actually, read must be called several times. Are you worried by the
cost of the indirect function calls?
> Plus in your last commit you make the huge mistake of preferring code
> elegance over performance, and resort to calling init, read, and free
> *twice*, the first just to read *two* bytes (yes, 2 bytes). Why do
> you think the code was coded that way in the first place? A fit of
> absent mindedness perhaps? No, for performance reasons.
I also pointed this out, and that's mosly why I created a separate last
patch. I don't think it's optimal, but replicating the metadata length
handling in the backends isn't either. There's RFC in the subject after
all...
> You've totally broken all the read optimisations in zlib decompression
> for block devices. Buffer_heads are deliberately passed directly to
> the zlib decompressor to allow it to optimise performance (pass the
> buffer_head directly to zlib etc.). Buffer_heads are exposed to the
> decompressors without crappy go-between wrappers deliberately.
At least this is something I noticed and expressed my concers about.
You probably missed that due to the lots of scrolling required by my
sub-par patch-posting technique. :)
> Unfortunately there are lots of other instances where you've
> eliminated carefully optimised code.
Could you please provide some examples?
> In short this doesn't pass my quality threshold or the make the
> minimum changes necessary threshold. I wouldn't consider asking Linus
> to merge this.
Please don't. It's a proof of concept, nothing more. Even a proof of a
wrong concept.
> BTW as a comparison, I have added MTD support to Squashfs twice in the
> past, *with* bad block support.
Unfortunately you aren't allowed to share those, if I read it right.
> The latest has a diff patch of ~500 lines, with far less complexity
> and code changes necessary.
Sure, have a look at my very first patch: 3 files changed, 210
insertions, 21 deletions. And that didn't remove bdev support.
> BTW2 as evidenced by your FIXME comments against my code in your patch,
> you really have to get over the notion that if you don't understand it,
> the code must be wrong.
Sometimes those are FIXMEs to make finding them easier, and are
questions really. If you feel like answering them:
/* FIXME: == s->s_blocksize(_bits)? */
Why do you use devblksize and devblksize_log2 in squashfs_sb_info?
Aren't they the same as sb->s_blocksize and sb->s_blocksize_bits, as
their names suggest?
/* FIXME: squashfs_put_super has meta_index instead */
The failed_mount: part of squashfs_fill_super() is very similar to
squashfs_put_super(), but frees meta_index instead of id_table. Why?
These are probably easy questions for people knowing the VFS layer well,
but I am not one of those. They just triggered my built-in pattern
matcher.
--
Regards,
Feri.
Ferenc Wagner
fs/squashfs/Kconfig | 1 +
fs/squashfs/Makefile | 1 +
fs/squashfs/backend.c | 15 ++++
fs/squashfs/mtd.c | 179 ++++++++++++++++++++++++++++++++++++++++++++++++
fs/squashfs/squashfs.h | 4 +
5 files changed, 200 insertions(+), 0 deletions(-)
create mode 100644 fs/squashfs/mtd.c
diff --git a/fs/squashfs/Kconfig b/fs/squashfs/Kconfig
index 40a3f15..6849e70 100644
--- a/fs/squashfs/Kconfig
+++ b/fs/squashfs/Kconfig
@@ -1,5 +1,6 @@
config SQUASHFS
tristate "SquashFS 4.0 - Squashed file system support"
+ depends on BLOCK || MTD
select ZLIB_INFLATE
help
Saying Y here includes support for SquashFS 4.0 (a Compressed
diff --git a/fs/squashfs/Makefile b/fs/squashfs/Makefile
index 80f1cbe..8d5c0b8 100644
--- a/fs/squashfs/Makefile
+++ b/fs/squashfs/Makefile
@@ -7,3 +7,4 @@ squashfs-y += cache.o dir.o export.o file.o fragment.o id.o inode.o
squashfs-y += namei.o super.o symlink.o zlib_wrapper.o decompressor.o backend.o
squashfs-$(CONFIG_SQUASHFS_LZMA) += lzma_wrapper.o
squashfs-$(CONFIG_BLOCK) += block.o
+squashfs-$(CONFIG_MTD) += mtd.o
diff --git a/fs/squashfs/backend.c b/fs/squashfs/backend.c
index b83a5e2..a6136ca 100644
--- a/fs/squashfs/backend.c
+++ b/fs/squashfs/backend.c
@@ -1,6 +1,7 @@
#include <linux/fs.h>
#include <linux/slab.h>
#include <linux/buffer_head.h>
+#include <linux/mtd/super.h>
#include "squashfs_fs_i.h"
#include "squashfs.h"
@@ -13,6 +14,10 @@ int squashfs_find_backend(struct file_system_type *fs_type, int flags,
if (!get_sb_bdev(fs_type, flags, dev_name, data, fill_bdev_super, mnt))
return 0;
#endif
+#ifdef CONFIG_MTD
+ if (!get_sb_mtd(fs_type, flags, dev_name, data, fill_mtd_super, mnt))
+ return 0;
+#endif
WARNING("no suitable backend found\n");
return -EINVAL;
}
@@ -25,6 +30,12 @@ void squashfs_kill_super(struct super_block *sb)
return;
}
#endif
+#ifdef CONFIG_MTD
+ if (sb->s_mtd) {
+ kill_mtd_super(sb);
+ return;
+ }
+#endif
ERROR("squashfs_kill_super: no device behind the super block\n");
}
@@ -43,6 +54,10 @@ int squashfs_read_data(struct super_block *sb, void **buffer, u64 index,
if (sb->s_bdev)
return bdev_read_data(sb, buffer, index, length, next_index, srclength, pages);
#endif
+#ifdef CONFIG_MTD
+ if (sb->s_mtd)
+ return mtd_read_data(sb, buffer, index, length, next_index, srclength, pages);
+#endif
ERROR("squashfs_read_data: no device behind the super block\n");
return -EIO;
}
diff --git a/fs/squashfs/mtd.c b/fs/squashfs/mtd.c
new file mode 100644
index 0000000..b067616
--- /dev/null
+++ b/fs/squashfs/mtd.c
@@ -0,0 +1,179 @@
+/*
+ * mtd.c
+ */
+
+/*
+ * This file implements the low-level routines to read and decompress
+ * datablocks and metadata blocks from an MTD.
+ */
+
+#include <linux/fs.h>
+#include <linux/vfs.h>
+#include <linux/slab.h>
+#include <linux/string.h>
+#include <linux/buffer_head.h>
+#include <linux/mtd/mtd.h>
+
+#include "squashfs_fs.h"
+#include "squashfs_fs_sb.h"
+#include "squashfs_fs_i.h"
+#include "squashfs.h"
+#include "decompressor.h"
+
+static int checked_mtd_read(struct mtd_info *mi, u64 index, int length,
+ void *buf)
+{
+ int ret, retlen;
+
+ TRACE("Entering checked_mtd_read: index=0x%llx, length=%d\n",
+ index, length);
+ ret = mi->read(mi, index, length, &retlen, buf);
+ if (ret) {
+ if (ret == -EUCLEAN || ret == -EBADMSG)
+ WARNING("checked_mtd_read(index=0x%llx, length=%d): "
+ "recoverable error %d\n", index, length, ret);
+ else {
+ ERROR("checked_mtd_read(index=0x%llx, length=%d): %d\n",
+ index, length, ret);
+ return ret;
+ }
+ }
+ if (retlen != length) {
+ ERROR("checked_mtd_read(index=0x%llx, length=%d) short read: %d\n",
+ index, length, retlen);
+ return -EIO;
+ }
+ return 0;
+}
+
+static int update_buffer(struct buffer_head *bh)
+{
+ struct mtd_info *mi = (struct mtd_info *)bh->b_bdev;
+ int ret = checked_mtd_read(mi, bh->b_blocknr, bh->b_size, bh->b_data);
+ if (ret)
+ return 0;
+ return 1;
+}
+
+static void put_buffer(struct buffer_head *bh)
+{
+}
+
+/*
+ * Big buffer_heads require more memory, but if a single one is enough,
+ * that can be special-cased in unlzma to avoid the extra memcpy.
+ * A better unlzma interface would be preferable, though.
+ */
+int mtd_read_data(struct super_block *sb, void **buffer, u64 index,
+ int length, u64 *next_index, int srclength, int pages)
+{
+ struct squashfs_sb_info *msblk = sb->s_fs_info;
+ struct mtd_info *mi = sb->s_mtd;
+ u64 i = index;
+ int bytes_left, compressed;
+
+ if (length) { /* Data block */
+ compressed = SQUASHFS_COMPRESSED_BLOCK(length);
+ length = SQUASHFS_COMPRESSED_SIZE_BLOCK(length);
+
+ TRACE("Block @ 0x%llx, %scompressed size %d, src size %d\n",
+ index, compressed ? "" : "un", length, srclength);
+ } else { /* Metadata block */
+ u16 metalen;
+ if ((index + 2) > msblk->bytes_used)
+ goto read_failure;
+ if (checked_mtd_read(mi, index, 2, &metalen))
+ goto read_failure;
+ i += 2;
+ length = le16_to_cpu(metalen);
+ compressed = SQUASHFS_COMPRESSED(length);
+ length = SQUASHFS_COMPRESSED_SIZE(length);
+
+ TRACE("Block @ 0x%llx, %scompressed size %d\n", index,
+ compressed ? "" : "un", length);
+ }
+ if (next_index)
+ *next_index = i + length;
+
+ if (length < 0 || length > srclength || i + length > msblk->bytes_used)
+ goto read_failure;
+
+ if (compressed) {
+ struct buffer_head **bh, *bhs;
+ int bh_num = (max_t(int, msblk->block_size, SQUASHFS_METADATA_SIZE) >>
+ msblk->devblksize_log2) + 1;
+ u_char *data;
+ int b;
+
+ bh = kmalloc(bh_num * sizeof(*bh), GFP_KERNEL);
+ if (bh == NULL)
+ return -ENOMEM;
+ bhs = kmalloc(bh_num * sizeof(*bhs), GFP_KERNEL);
+ if (bhs == NULL) {
+ kfree(bh);
+ return -ENOMEM;
+ }
+ data = kmalloc(msblk->devblksize, GFP_KERNEL);
+ if (data == NULL) {
+ kfree(bhs);
+ kfree(bh);
+ return -ENOMEM;
+ }
+
+ bytes_left = length;
+ for (b = 0; bytes_left > 0; b++) {
+ bh[b] = &bhs[b];
+ bhs[b].b_blocknr = i;
+ bhs[b].b_size = min(msblk->devblksize, bytes_left);
+ /* We know that the decompressors will use each buffer_head
+ * only once, so update_buffer may change the data under them. */
+ bhs[b].b_data = data;
+ bhs[b].b_bdev = (void *)mi;
+ i += msblk->devblksize;
+ bytes_left -= msblk->devblksize;
+ }
+
+ length = squashfs_decompress(msblk, buffer, bh, b, 0,
+ length, srclength, pages, update_buffer, put_buffer);
+ if (length < 0) {
+ kfree(data);
+ kfree(bhs);
+ kfree(bh);
+ goto read_failure;
+ }
+ } else { /* Not compressed */
+ int page = 0;
+ bytes_left = length;
+ while (bytes_left > 0) {
+ int blk = min_t(int, bytes_left, PAGE_CACHE_SIZE);
+ if (checked_mtd_read(mi, i, blk, buffer[page++]))
+ goto read_failure;
+ bytes_left -= blk;
+ i += blk;
+ }
+ }
+ return length;
+
+read_failure:
+ ERROR("mtd_read_data failed to read block 0x%llx\n",
+ (unsigned long long) index);
+ return -EIO;
+}
+
+int fill_mtd_super(struct super_block *sb, void *data, int silent)
+{
+ struct squashfs_sb_info *msblk;
+ char b[BDEVNAME_SIZE];
+
+ TRACE("Entering fill_mtd_super\n");
+
+ msblk = kzalloc(sizeof(*msblk), GFP_KERNEL);
+ if (!msblk)
+ return -ENOMEM;
+
+ sb->s_fs_info = msblk;
+ msblk->devblksize = PAGE_CACHE_SIZE;
+ msblk->devblksize_log2 = PAGE_CACHE_SHIFT;
+ snprintf(b, sizeof b, "mtd%d", sb->s_mtd->index);
+ return squashfs_fill_super(sb, data, silent, b, sb->s_mtd->size);
+}
diff --git a/fs/squashfs/squashfs.h b/fs/squashfs/squashfs.h
index 7c5cd72..05a85e2 100644
--- a/fs/squashfs/squashfs.h
+++ b/fs/squashfs/squashfs.h
@@ -105,6 +105,10 @@ extern const struct squashfs_decompressor squashfs_lzma_comp_ops;
extern int fill_bdev_super(struct super_block *, void *, int);
extern int bdev_read_data(struct super_block *, void **, u64, int, u64 *, int, int);
+/* mtd.c */
+extern int fill_mtd_super(struct super_block *, void *, int);
+extern int mtd_read_data(struct super_block *, void **, u64, int, u64 *, int, int);
+
#ifndef CONFIG_BLOCK
struct buffer_head {
sector_t b_blocknr;
--
1.6.5
Ferenc Wagner
so provide a basic definition even if CONFIG_BLOCK=n (which results in
a backendless - ie. useless - squashfs right now).
---
fs/squashfs/Kconfig | 1 -
fs/squashfs/Makefile | 5 +++--
fs/squashfs/backend.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
fs/squashfs/backend.h | 12 ++++++++++++
fs/squashfs/block.c | 33 +++++++++++++++++++++------------
fs/squashfs/squashfs.h | 18 +++++++++++++++++-
fs/squashfs/super.c | 45 ++++++++++-----------------------------------
7 files changed, 111 insertions(+), 51 deletions(-)
create mode 100644 fs/squashfs/backend.c
create mode 100644 fs/squashfs/backend.h
diff --git a/fs/squashfs/Kconfig b/fs/squashfs/Kconfig
index 7ec5d7e..40a3f15 100644
--- a/fs/squashfs/Kconfig
+++ b/fs/squashfs/Kconfig
@@ -1,6 +1,5 @@
config SQUASHFS
tristate "SquashFS 4.0 - Squashed file system support"
- depends on BLOCK
select ZLIB_INFLATE
help
Saying Y here includes support for SquashFS 4.0 (a Compressed
diff --git a/fs/squashfs/Makefile b/fs/squashfs/Makefile
index 45aaefd..80f1cbe 100644
--- a/fs/squashfs/Makefile
+++ b/fs/squashfs/Makefile
@@ -3,6 +3,7 @@
#
obj-$(CONFIG_SQUASHFS) += squashfs.o
-squashfs-y += block.o cache.o dir.o export.o file.o fragment.o id.o inode.o
-squashfs-y += namei.o super.o symlink.o zlib_wrapper.o decompressor.o
+squashfs-y += cache.o dir.o export.o file.o fragment.o id.o inode.o
+squashfs-y += namei.o super.o symlink.o zlib_wrapper.o decompressor.o backend.o
squashfs-$(CONFIG_SQUASHFS_LZMA) += lzma_wrapper.o
+squashfs-$(CONFIG_BLOCK) += block.o
diff --git a/fs/squashfs/backend.c b/fs/squashfs/backend.c
new file mode 100644
index 0000000..b83a5e2
--- /dev/null
+++ b/fs/squashfs/backend.c
@@ -0,0 +1,48 @@
+#include <linux/fs.h>
+#include <linux/slab.h>
+#include <linux/buffer_head.h>
+
+#include "squashfs_fs_i.h"
+#include "squashfs.h"
+#include "backend.h"
+
+int squashfs_find_backend(struct file_system_type *fs_type, int flags,
+ const char *dev_name, void *data, struct vfsmount *mnt)
+{
+#ifdef CONFIG_BLOCK
+ if (!get_sb_bdev(fs_type, flags, dev_name, data, fill_bdev_super, mnt))
+ return 0;
+#endif
+ WARNING("no suitable backend found\n");
+ return -EINVAL;
+}
+
+void squashfs_kill_super(struct super_block *sb)
+{
+#ifdef CONFIG_BLOCK
+ if (sb->s_bdev) {
+ kill_block_super(sb);
+ return;
+ }
+#endif
+ ERROR("squashfs_kill_super: no device behind the super block\n");
+}
+
+/*
+ * Read and decompress a metadata block or datablock. Length is non-zero
+ * if a datablock is being read (the size is stored elsewhere in the
+ * filesystem), otherwise the length is obtained from the first two bytes of
+ * the metadata block. A bit in the length field indicates if the block
+ * is stored uncompressed in the filesystem (usually because compression
+ * generated a larger block - this does occasionally happen with zlib).
+ */
+int squashfs_read_data(struct super_block *sb, void **buffer, u64 index,
+ int length, u64 *next_index, int srclength, int pages)
+{
+#ifdef CONFIG_BLOCK
+ if (sb->s_bdev)
+ return bdev_read_data(sb, buffer, index, length, next_index, srclength, pages);
+#endif
+ ERROR("squashfs_read_data: no device behind the super block\n");
+ return -EIO;
+}
diff --git a/fs/squashfs/backend.h b/fs/squashfs/backend.h
new file mode 100644
index 0000000..54c973e
--- /dev/null
+++ b/fs/squashfs/backend.h
@@ -0,0 +1,12 @@
+#ifndef BACKEND_H
+#define BACKEND_H
+
+#include <linux/fs.h>
+#include <linux/vfs.h>
+
+int squashfs_find_backend(struct file_system_type *, int, const char *,
+ void *, struct vfsmount *);
+void squashfs_kill_super(struct super_block *);
+int squashfs_read_data(struct super_block *, void **, u64, int, u64 *, int, int);
+
+#endif
diff --git a/fs/squashfs/block.c b/fs/squashfs/block.c
index c7e5881..f16bf27 100644
--- a/fs/squashfs/block.c
+++ b/fs/squashfs/block.c
@@ -23,7 +23,7 @@
/*
* This file implements the low-level routines to read and decompress
- * datablocks and metadata blocks.
+ * datablocks and metadata blocks from a block device.
*/
#include <linux/fs.h>
@@ -75,16 +75,7 @@ static struct buffer_head *get_block_length(struct super_block *sb,
return bh;
}
-
-/*
- * Read and decompress a metadata block or datablock. Length is non-zero
- * if a datablock is being read (the size is stored elsewhere in the
- * filesystem), otherwise the length is obtained from the first two bytes of
- * the metadata block. A bit in the length field indicates if the block
- * is stored uncompressed in the filesystem (usually because compression
- * generated a larger block - this does occasionally happen with zlib).
- */
-int squashfs_read_data(struct super_block *sb, void **buffer, u64 index,
+int bdev_read_data(struct super_block *sb, void **buffer, u64 index,
int length, u64 *next_index, int srclength, int pages)
{
struct squashfs_sb_info *msblk = sb->s_fs_info;
@@ -203,8 +194,26 @@ block_release:
put_bh(bh[k]);
read_failure:
- ERROR("squashfs_read_data failed to read block 0x%llx\n",
+ ERROR("bdev_read_data failed to read block 0x%llx\n",
(unsigned long long) index);
kfree(bh);
return -EIO;
}
+
+int fill_bdev_super(struct super_block *sb, void *data, int silent)
+{
+ struct squashfs_sb_info *msblk;
+ char b[BDEVNAME_SIZE];
+
+ TRACE("Entering fill_bdev_super\n");
+
+ msblk = kzalloc(sizeof(*msblk), GFP_KERNEL);
+ if (!msblk)
+ return -ENOMEM;
+
+ sb->s_fs_info = msblk;
+ msblk->devblksize = sb_min_blocksize(sb, BLOCK_SIZE);
+ msblk->devblksize_log2 = ffz(~msblk->devblksize);
+ return squashfs_fill_super(sb, data, silent, bdevname(sb->s_bdev, b),
+ i_size_read(sb->s_bdev->bd_inode));
+}
diff --git a/fs/squashfs/squashfs.h b/fs/squashfs/squashfs.h
index d094886..7c5cd72 100644
--- a/fs/squashfs/squashfs.h
+++ b/fs/squashfs/squashfs.h
@@ -32,7 +32,7 @@ static inline struct squashfs_inode_info *squashfs_i(struct inode *inode)
return list_entry(inode, struct squashfs_inode_info, vfs_inode);
}
-/* block.c */
+/* backend.c */
extern int squashfs_read_data(struct super_block *, void **, u64, int, u64 *,
int, int);
@@ -73,6 +73,9 @@ extern struct inode *squashfs_iget(struct super_block *, long long,
unsigned int);
extern int squashfs_read_inode(struct inode *, long long);
+/* super.c */
+extern int squashfs_fill_super(struct super_block *, void *, int, const char *, int);
+
/*
* Inodes, files and decompressor operations
*/
@@ -97,3 +100,16 @@ extern const struct squashfs_decompressor squashfs_zlib_comp_ops;
/* lzma wrapper.c */
extern const struct squashfs_decompressor squashfs_lzma_comp_ops;
+
+/* block.c */
+extern int fill_bdev_super(struct super_block *, void *, int);
+extern int bdev_read_data(struct super_block *, void **, u64, int, u64 *, int, int);
+
+#ifndef CONFIG_BLOCK
+struct buffer_head {
+ sector_t b_blocknr;
+ size_t b_size;
+ char *b_data;
+ void *b_bdev;
+};
+#endif
diff --git a/fs/squashfs/super.c b/fs/squashfs/super.c
index 3550aec..7c896ca 100644
--- a/fs/squashfs/super.c
+++ b/fs/squashfs/super.c
@@ -42,6 +42,7 @@
#include "squashfs_fs_i.h"
#include "squashfs.h"
#include "decompressor.h"
+#include "backend.h"
static struct file_system_type squashfs_fs_type;
static const struct super_operations squashfs_super_ops;
@@ -73,11 +74,11 @@ static const struct squashfs_decompressor *supported_squashfs_filesystem(short
}
-static int squashfs_fill_super(struct super_block *sb, void *data, int silent)
+int squashfs_fill_super(struct super_block *sb, void *data, int silent,
+ const char *name, int size)
{
- struct squashfs_sb_info *msblk;
+ struct squashfs_sb_info *msblk = sb->s_fs_info;
struct squashfs_super_block *sblk = NULL;
- char b[BDEVNAME_SIZE];
struct inode *root;
long long root_inode;
unsigned short flags;
@@ -87,22 +88,12 @@ static int squashfs_fill_super(struct super_block *sb, void *data, int silent)
TRACE("Entered squashfs_fill_superblock\n");
- sb->s_fs_info = kzalloc(sizeof(*msblk), GFP_KERNEL);
- if (sb->s_fs_info == NULL) {
- ERROR("Failed to allocate squashfs_sb_info\n");
- return -ENOMEM;
- }
- msblk = sb->s_fs_info;
-
sblk = kzalloc(sizeof(*sblk), GFP_KERNEL);
if (sblk == NULL) {
ERROR("Failed to allocate squashfs_super_block\n");
- goto failure;
+ return -ENOMEM;
}
- msblk->devblksize = sb_min_blocksize(sb, BLOCK_SIZE);
- msblk->devblksize_log2 = ffz(~msblk->devblksize);
-
mutex_init(&msblk->read_data_mutex);
mutex_init(&msblk->meta_index_mutex);
@@ -126,8 +117,7 @@ static int squashfs_fill_super(struct super_block *sb, void *data, int silent)
sb->s_magic = le32_to_cpu(sblk->s_magic);
if (sb->s_magic != SQUASHFS_MAGIC) {
if (!silent)
- ERROR("Can't find a SQUASHFS superblock on %s\n",
- bdevname(sb->s_bdev, b));
+ ERROR("Can't find a SQUASHFS superblock on %s\n", name);
goto failed_mount;
}
@@ -149,8 +139,7 @@ static int squashfs_fill_super(struct super_block *sb, void *data, int silent)
/* Check the filesystem does not extend beyond the end of the
block device */
msblk->bytes_used = le64_to_cpu(sblk->bytes_used);
- if (msblk->bytes_used < 0 || msblk->bytes_used >
- i_size_read(sb->s_bdev->bd_inode))
+ if (msblk->bytes_used < 0 || msblk->bytes_used > size)
goto failed_mount;
/* Check block size for sanity */
@@ -182,7 +171,7 @@ static int squashfs_fill_super(struct super_block *sb, void *data, int silent)
msblk->inodes = le32_to_cpu(sblk->inodes);
flags = le16_to_cpu(sblk->flags);
- TRACE("Found valid superblock on %s\n", bdevname(sb->s_bdev, b));
+ TRACE("Found valid superblock on %s\n", name);
TRACE("Inodes are %scompressed\n", SQUASHFS_UNCOMPRESSED_INODES(flags)
? "un" : "");
TRACE("Data is %scompressed\n", SQUASHFS_UNCOMPRESSED_DATA(flags)
@@ -304,11 +293,6 @@ failed_mount:
sb->s_fs_info = NULL;
kfree(sblk);
return err;
-
-failure:
- kfree(sb->s_fs_info);
- sb->s_fs_info = NULL;
- return -ENOMEM;
}
@@ -361,15 +345,6 @@ static void squashfs_put_super(struct super_block *sb)
}
-static int squashfs_get_sb(struct file_system_type *fs_type, int flags,
- const char *dev_name, void *data,
- struct vfsmount *mnt)
-{
- return get_sb_bdev(fs_type, flags, dev_name, data, squashfs_fill_super,
- mnt);
-}
-
-
static struct kmem_cache *squashfs_inode_cachep;
@@ -442,8 +417,8 @@ static void squashfs_destroy_inode(struct inode *inode)
static struct file_system_type squashfs_fs_type = {
.owner = THIS_MODULE,
.name = "squashfs",
- .get_sb = squashfs_get_sb,
- .kill_sb = kill_block_super,
+ .get_sb = squashfs_find_backend,
+ .kill_sb = squashfs_kill_super,
.fs_flags = FS_REQUIRES_DEV
};
--
1.6.5
Ferenc Wagner
Here is another, very different take on the subject. It isn't a real
plugin system, but touches as little existing code as possible, and
still quite easy to extend if needed. The MTD with LZMA case is somewhat
challenged, as the current interface mismatch requires extra copying.
It seems fairly easy to change the kernel decompressor interface to
scatter/gather operation in the future, to do away with this.
This patch does not consider SQUASHFS_METADATA_SIZE during buffer_head
allocation in block.c, on the principle of hands-off.
Extra question: does squashfs_put_super require BKL protection?
Comments welcome.
Regards,
Feri.
Ferenc Wagner (3):
squashfs: parametrize decompressors on buffer_head operations
squashfs: gather everything block device specific into block.c
squashfs: add MTD backend
fs/squashfs/Kconfig | 2 +-
fs/squashfs/Makefile | 6 +-
fs/squashfs/backend.c | 63 +++++++++++++++
fs/squashfs/backend.h | 12 +++
fs/squashfs/block.c | 42 +++++++---
fs/squashfs/decompressor.h | 8 +-
fs/squashfs/lzma_wrapper.c | 11 ++-
fs/squashfs/mtd.c | 179 ++++++++++++++++++++++++++++++++++++++++++++
fs/squashfs/squashfs.h | 22 +++++-
fs/squashfs/super.c | 45 +++---------
fs/squashfs/zlib_wrapper.c | 13 ++--
11 files changed, 337 insertions(+), 66 deletions(-)
create mode 100644 fs/squashfs/backend.c
create mode 100644 fs/squashfs/backend.h
create mode 100644 fs/squashfs/mtd.c
Ferenc Wagner
fs/squashfs/block.c | 9 ++++++++-
fs/squashfs/decompressor.h | 8 +++++---
fs/squashfs/lzma_wrapper.c | 11 ++++++-----
fs/squashfs/zlib_wrapper.c | 13 +++++++------
4 files changed, 26 insertions(+), 15 deletions(-)
diff --git a/fs/squashfs/block.c b/fs/squashfs/block.c
index 6f9914d..c7e5881 100644
--- a/fs/squashfs/block.c
+++ b/fs/squashfs/block.c
@@ -37,6 +37,13 @@
#include "squashfs_fs_i.h"
#include "squashfs.h"
#include "decompressor.h"
+
+static int update_buffer(struct buffer_head *bh)
+{
+ wait_on_buffer(bh);
+ return buffer_uptodate(bh);
+}
+
/*
* Read the metadata block length, this is stored in the first two
* bytes of the metadata block.
@@ -152,7 +159,7 @@ int squashfs_read_data(struct super_block *sb, void **buffer, u64 index,
if (compressed) {
length = squashfs_decompress(msblk, buffer, bh, b, offset,
- length, srclength, pages);
+ length, srclength, pages, update_buffer, put_bh);
if (length < 0)
goto read_failure;
} else {
diff --git a/fs/squashfs/decompressor.h b/fs/squashfs/decompressor.h
index 7425f80..7de948b 100644
--- a/fs/squashfs/decompressor.h
+++ b/fs/squashfs/decompressor.h
@@ -27,7 +27,8 @@ struct squashfs_decompressor {
void *(*init)(struct squashfs_sb_info *);
void (*free)(void *);
int (*decompress)(struct squashfs_sb_info *, void **,
- struct buffer_head **, int, int, int, int, int);
+ struct buffer_head **, int, int, int, int, int,
+ int (*)(struct buffer_head *), void (*)(struct buffer_head *));
int id;
char *name;
int supported;
@@ -47,9 +48,10 @@ static inline void squashfs_decompressor_free(struct squashfs_sb_info *msblk,
static inline int squashfs_decompress(struct squashfs_sb_info *msblk,
void **buffer, struct buffer_head **bh, int b, int offset, int length,
- int srclength, int pages)
+ int srclength, int pages,
+ int (*update_buffer)(struct buffer_head *), void (*put_buffer)(struct buffer_head *))
{
return msblk->decompressor->decompress(msblk, buffer, bh, b, offset,
- length, srclength, pages);
+ length, srclength, pages, update_buffer, put_buffer);
}
#endif
diff --git a/fs/squashfs/lzma_wrapper.c b/fs/squashfs/lzma_wrapper.c
index 9fa617d..994c37e 100644
--- a/fs/squashfs/lzma_wrapper.c
+++ b/fs/squashfs/lzma_wrapper.c
@@ -88,7 +88,9 @@ static void lzma_free(void *strm)
static int lzma_uncompress(struct squashfs_sb_info *msblk, void **buffer,
struct buffer_head **bh, int b, int offset, int length, int srclength,
- int pages)
+ int pages,
+ int (*update_buffer)(struct buffer_head *),
+ void (*put_buffer)(struct buffer_head *))
{
struct squashfs_lzma *stream = msblk->stream;
void *buff = stream->input;
@@ -97,8 +99,7 @@ static int lzma_uncompress(struct squashfs_sb_info *msblk, void **buffer,
mutex_lock(&lzma_mutex);
for (i = 0; i < b; i++) {
- wait_on_buffer(bh[i]);
- if (!buffer_uptodate(bh[i]))
+ if (!update_buffer(bh[i]))
goto block_release;
avail = min(bytes, msblk->devblksize - offset);
@@ -106,7 +107,7 @@ static int lzma_uncompress(struct squashfs_sb_info *msblk, void **buffer,
buff += avail;
bytes -= avail;
offset = 0;
- put_bh(bh[i]);
+ put_buffer(bh[i]);
}
lzma_error = 0;
@@ -131,7 +132,7 @@ static int lzma_uncompress(struct squashfs_sb_info *msblk, void **buffer,
block_release:
for (; i < b; i++)
- put_bh(bh[i]);
+ put_buffer(bh[i]);
failed:
mutex_unlock(&lzma_mutex);
diff --git a/fs/squashfs/zlib_wrapper.c b/fs/squashfs/zlib_wrapper.c
index 4dd70e0..09cdcc1 100644
--- a/fs/squashfs/zlib_wrapper.c
+++ b/fs/squashfs/zlib_wrapper.c
@@ -63,7 +63,9 @@ static void zlib_free(void *strm)
static int zlib_uncompress(struct squashfs_sb_info *msblk, void **buffer,
struct buffer_head **bh, int b, int offset, int length, int srclength,
- int pages)
+ int pages,
+ int (*update_buffer)(struct buffer_head *),
+ void (*put_buffer)(struct buffer_head *))
{
int zlib_err = 0, zlib_init = 0;
int avail, bytes, k = 0, page = 0;
@@ -79,13 +81,12 @@ static int zlib_uncompress(struct squashfs_sb_info *msblk, void **buffer,
if (stream->avail_in == 0 && k < b) {
avail = min(bytes, msblk->devblksize - offset);
bytes -= avail;
- wait_on_buffer(bh[k]);
- if (!buffer_uptodate(bh[k]))
+ if (!update_buffer(bh[k]))
goto release_mutex;
if (avail == 0) {
offset = 0;
- put_bh(bh[k++]);
+ put_buffer(bh[k++]);
continue;
}
@@ -113,7 +114,7 @@ static int zlib_uncompress(struct squashfs_sb_info *msblk, void **buffer,
zlib_err = zlib_inflate(stream, Z_SYNC_FLUSH);
if (stream->avail_in == 0 && k < b)
- put_bh(bh[k++]);
+ put_buffer(bh[k++]);
} while (zlib_err == Z_OK);
if (zlib_err != Z_STREAM_END) {
@@ -134,7 +135,7 @@ release_mutex:
mutex_unlock(&msblk->read_data_mutex);
for (; k < b; k++)
- put_bh(bh[k]);
+ put_buffer(bh[k]);
return -EIO;
}
--
1.6.5
Ferenc Wagner
I lost this free somehow. Does squashfs_put_super() need protection
from multiple invocation? If not, this kfree could be removed from
there, and the condition from both places.
fs/squashfs/backend.c | 4 ++++
1 files changed, 4 insertions(+), 0 deletions(-)
diff --git a/fs/squashfs/backend.c b/fs/squashfs/backend.c
index a6136ca..0ae37ab 100644
--- a/fs/squashfs/backend.c
+++ b/fs/squashfs/backend.c
@@ -24,6 +24,10 @@ int squashfs_find_backend(struct file_system_type *fs_type, int flags,
void squashfs_kill_super(struct super_block *sb)
{
+ if (sb->s_fs_info) {
+ kfree(sb->s_fs_info);
+ sb->s_fs_info = NULL;
+ }
#ifdef CONFIG_BLOCK
if (sb->s_bdev) {
kill_block_super(sb);
Marco Stornelli
>
> Extra question: does squashfs_put_super require BKL protection?
>
We should get rid off of BKL, so if you need a lock, you should use a
local mutex.
Marco