[PATCH] scripts: add extract-vmlinux
Corentin Chary
kernel image (bzImage, etc..). It's inspired from (a subset of)
extract-ikconfig.
I don't know if this scripts really have this place in the kernel
tree, but it's something a lot of people have been looking for (mainly
people with xen < 4 that doesn't support bzImages at all).
Signed-off-by: Corentin Chary <corenti...@gmail.com>
---
scripts/extract-vmlinux | 60 +++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 60 insertions(+), 0 deletions(-)
create mode 100755 scripts/extract-vmlinux
diff --git a/scripts/extract-vmlinux b/scripts/extract-vmlinux
new file mode 100755
index 0000000..444f404
--- /dev/null
+++ b/scripts/extract-vmlinux
@@ -0,0 +1,60 @@
+#!/bin/sh
+# ----------------------------------------------------------------------
+# extract-vmlinux - Extract uncompressed vmlinux from a kernel image
+#
+# The obscure use of the "tr" filter is to work around older versions of
+# "grep" that report the byte offset of the line instead of the pattern.
+#
+# Inspired from extract-ikconfig
+# (c) 2009,2010 Dick Streefland <di...@streefland.net>
+#
+# (c) 2011 Corentin Chary <corenti...@gmail.com>
+# Licensed under the terms of the GNU General Public License.
+# ----------------------------------------------------------------------
+
+check_elf()
+{
+ # Use readelf to check if it's a valid ELF
+ # There is probably a better way to do that...
+ readelf -h $1 > /dev/null 2>&1 || return 1
+
+ cat $1
+ exit 0
+}
+
+try_decompress()
+{
+ for pos in `tr "$1\n$2" "\n$2=" < "$img" | grep -abo "^$2"`
+ do
+ pos=${pos%%:*}
+ tail -c+$pos "$img" | $3 > $tmp 2> /dev/null
+ check_elf $tmp
+ done
+}
+
+# Check invocation:
+me=${0##*/}
+img=$1
+if [ $# -ne 1 -o ! -s "$img" ]
+then
+ echo "Usage: $me <kernel-image>" >&2
+ exit 2
+fi
+
+# Prepare temp files:
+tmp=$(mktemp /tmp/vmlinux-XXX)
+trap "rm -f $tmp" 0
+
+# Initial attempt for uncompressed images or objects:
+check_elf $img
+
+# That didn't work, so retry after decompression.
+try_decompress '\037\213\010' xy gunzip
+try_decompress '\3757zXZ\000' abcde unxz
+try_decompress 'BZh' xy bunzip2
+try_decompress '\135\0\0\0' xxx unlzma
+try_decompress '\211\114\132' xy 'lzop -d'
+
+# Bail out:
+echo "$me: Cannot find vmlinux." >&2
+exit 1
--
1.7.3.4
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Michal Marek
> This script can be used to extract vmlinux from a compressed
> kernel image (bzImage, etc..). It's inspired from (a subset of)
> extract-ikconfig.
Nice.
> I don't know if this scripts really have this place in the kernel
> tree, but it's something a lot of people have been looking for (mainly
> people with xen < 4 that doesn't support bzImages at all).
I think it makes sense to have it in the tree.
> Signed-off-by: Corentin Chary <corenti...@gmail.com>
> ---
> scripts/extract-vmlinux | 60 +++++++++++++++++++++++++++++++++++++++++++++++
> 1 files changed, 60 insertions(+), 0 deletions(-)
> create mode 100755 scripts/extract-vmlinux
>
> diff --git a/scripts/extract-vmlinux b/scripts/extract-vmlinux
> new file mode 100755
> index 0000000..444f404
> --- /dev/null
> +++ b/scripts/extract-vmlinux
> @@ -0,0 +1,60 @@
> +#!/bin/sh
> +# ----------------------------------------------------------------------
> +# extract-vmlinux - Extract uncompressed vmlinux from a kernel image
> +#
> +# The obscure use of the "tr" filter is to work around older versions of
> +# "grep" that report the byte offset of the line instead of the pattern.
> +#
> +# Inspired from extract-ikconfig
> +# (c) 2009,2010 Dick Streefland <di...@streefland.net>
> +#
> +# (c) 2011 Corentin Chary <corenti...@gmail.com>
> +# Licensed under the terms of the GNU General Public License.
I guess you want to use the version 2 of the GPL?
Michal
Corentin Chary
Since it's heavily based on extract-ikconfig I'm not sure I can change
the license.
CCing previous authors of extract-ikconfig to see if they are ok with that.
Thanks,
--
Corentin Chary
http://xf.iksaif.net
Dick Streefland
| 2011/8/10 Michal Marek <mma...@suse.cz>:
| > I guess you want to use the version 2 of the GPL?
|
| Since it's heavily based on extract-ikconfig I'm not sure I can change
| the license.
| CCing previous authors of extract-ikconfig to see if they are ok with that.
GPL v2 is fine with me. I think that is the default version for the
kernel anyway.
--
Dick
Américo Wang
<corenti...@gmail.com> wrote:
> 2011/8/10 Michal Marek <mma...@suse.cz>:
>>
>
> Since it's heavily based on extract-ikconfig I'm not sure I can change
> the license.
> CCing previous authors of extract-ikconfig to see if they are ok with that.
>
I don't remember I wrote any code of extract-ikconfig, I did review
the original patch. I am fine with this patch too.
Thanks!
Corentin Chary
kernel image (bzImage, etc..). It's inspired from (a subset of)
extract-ikconfig.
It's something a lot of people have been looking for (mainly
people with xen < 4 that doesn't support bzImages at all).
Signed-off-by: Corentin Chary <corenti...@gmail.com>
---
scripts/extract-vmlinux | 60 +++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 60 insertions(+), 0 deletions(-)
create mode 100755 scripts/extract-vmlinux
diff --git a/scripts/extract-vmlinux b/scripts/extract-vmlinux
new file mode 100755
index 0000000..615d31a
--- /dev/null
+++ b/scripts/extract-vmlinux
@@ -0,0 +1,60 @@
+#!/bin/sh
+# ----------------------------------------------------------------------
+# extract-vmlinux - Extract uncompressed vmlinux from a kernel image
+#
+# The obscure use of the "tr" filter is to work around older versions of
+# "grep" that report the byte offset of the line instead of the pattern.
+#
+# Inspired from extract-ikconfig
+# (c) 2009,2010 Dick Streefland <di...@streefland.net>
+#
+# (c) 2011 Corentin Chary <corenti...@gmail.com>
+#
+# Licensed under the GNU General Public License, version 2 (GPLv2).
Paul Bolle
> This script can be used to extract vmlinux from a compressed
> kernel image (bzImage, etc..). It's inspired from (a subset of)
> extract-ikconfig.
This is more specific than what this scripts actually does, isn't it? At
least when I tried to read this script my impression is that it does two
things:
- check whether the input file is a valid ELF file;
- if not; try to find a compressed ELF file somewhere in the input file.
There's no checking whether the input file is a kernel image and there's
no checking whether the found ELF file actually is was a, well, vmlinux.
Both checks are perhaps far from trivial. Anyhow, if that's correct this
should be made more clear. Perhaps the script should even be called
something like extract-elf.
> It's something a lot of people have been looking for (mainly
> people with xen < 4 that doesn't support bzImages at all).
>
> Signed-off-by: Corentin Chary <corenti...@gmail.com>
> [...]
> +try_decompress()
> +{
> + for pos in `tr "$1\n$2" "\n$2=" < "$img" | grep -abo "^$2"`
> + do
> + pos=${pos%%:*}
> + tail -c+$pos "$img" | $3 > $tmp 2> /dev/null
Perhaps a few comments on the above lines would be nice. Without those
comments I must guess you're finding compressed data somewhere in the
input file. It also seems you're looping through the entire input file.
Or are (sequences of) commands like the above considered obvious?
> + check_elf $tmp
> + done
> +}
> +
> [...]
> +# Initial attempt for uncompressed images or objects:
> +check_elf $img
> +
> +# That didn't work, so retry after decompression.
> +try_decompress '\037\213\010' xy gunzip
> +try_decompress '\3757zXZ\000' abcde unxz
> +try_decompress 'BZh' xy bunzip2
> +try_decompress '\135\0\0\0' xxx unlzma
> +try_decompress '\211\114\132' xy 'lzop -d'
Perhaps you could first test whether these commands are available before
running try_decompress() with them?
Paul Bolle
Corentin Chary
> On Thu, 2011-08-11 at 10:53 +0200, Corentin Chary wrote:
>> This script can be used to extract vmlinux from a compressed
>> kernel image (bzImage, etc..). It's inspired from (a subset of)
>> extract-ikconfig.
>
> This is more specific than what this scripts actually does, isn't it? At
> least when I tried to read this script my impression is that it does two
> things:
> - check whether the input file is a valid ELF file;
> - if not; try to find a compressed ELF file somewhere in the input file.
>
> There's no checking whether the input file is a kernel image and there's
> no checking whether the found ELF file actually is was a, well, vmlinux.
> Both checks are perhaps far from trivial. Anyhow, if that's correct this
> should be made more clear. Perhaps the script should even be called
> something like extract-elf.
Yep, I didn't found a quick way to check that the file is a valid
vmlinux and the function check_elf() has a configusing name.
Is that one better ?
check_vmlinux()
{
# Use readelf to check if it's a valid ELF
# TODO: find a better to way to check that it's really vmlinux
# and not just another elf
readelf -h $1 > /dev/null 2>&1 || return 1
cat $1
exit 0
}
If you have a good way to check that an elf is a valid vmlinux, I'd be
happy to implement it.
>> It's something a lot of people have been looking for (mainly
>> people with xen < 4 that doesn't support bzImages at all).
>>
>> Signed-off-by: Corentin Chary <corenti...@gmail.com>
>> [...]
>> +try_decompress()
>> +{
>> + for pos in `tr "$1\n$2" "\n$2=" < "$img" | grep -abo "^$2"`
>> + do
>> + pos=${pos%%:*}
>> + tail -c+$pos "$img" | $3 > $tmp 2> /dev/null
>
> Perhaps a few comments on the above lines would be nice. Without those
> comments I must guess you're finding compressed data somewhere in the
> input file. It also seems you're looping through the entire input file.
> Or are (sequences of) commands like the above considered obvious?
It's copied from extract-ikconfig, there was no comment in it, so I
assumed it's obvious.
Basically it uses brute force (tm) to find well known headers and try
to decompress from here.
>> + check_elf $tmp
>> + done
>> +}
>> +
>> [...]
>> +# Initial attempt for uncompressed images or objects:
>> +check_elf $img
>> +
>> +# That didn't work, so retry after decompression.
>> +try_decompress '\037\213\010' xy gunzip
>> +try_decompress '\3757zXZ\000' abcde unxz
>> +try_decompress 'BZh' xy bunzip2
>> +try_decompress '\135\0\0\0' xxx unlzma
>> +try_decompress '\211\114\132' xy 'lzop -d'
>
> Perhaps you could first test whether these commands are available before
> running try_decompress() with them?
Same as before, if it's ok for extract-ikconfig not to check the
command, then it's ok for me.
If it's not, then we should patch both.
Personnaly I think that the script is small enought so that someone
who try to use it and can't make it work will understand what's
happening.
Note that extract-ikconfig is used in the build system and is far more
"critical" that extract-vmlinux, and nobody complained before :).
--
Corentin Chary
http://xf.iksaif.net
Michal Marek
> On Thu, Aug 11, 2011 at 1:28 PM, Paul Bolle <peb...@tiscali.nl> wrote:
>> On Thu, 2011-08-11 at 10:53 +0200, Corentin Chary wrote:
>>> +try_decompress()
>>> +{
>>> + for pos in `tr "$1\n$2" "\n$2=" < "$img" | grep -abo "^$2"`
>>> + do
>>> + pos=${pos%%:*}
>>> + tail -c+$pos "$img" | $3 > $tmp 2> /dev/null
>>
>> Perhaps a few comments on the above lines would be nice. Without those
>> comments I must guess you're finding compressed data somewhere in the
>> input file. It also seems you're looping through the entire input file.
>> Or are (sequences of) commands like the above considered obvious?
>
> It's copied from extract-ikconfig, there was no comment in it, so I
> assumed it's obvious.
> Basically it uses brute force (tm) to find well known headers and try
> to decompress from here.
And explanation of the "tr" trick would not hurt though. It replaces the
binary magic with an ascii sequence on a new line (for older grep
versions to get it right) and tries all occurrences of that sequence, am
I right? But I wouldn't call it "obvious" :).
>>> +# That didn't work, so retry after decompression.
>>> +try_decompress '\037\213\010' xy gunzip
>>> +try_decompress '\3757zXZ\000' abcde unxz
>>> +try_decompress 'BZh' xy bunzip2
>>> +try_decompress '\135\0\0\0' xxx unlzma
>>> +try_decompress '\211\114\132' xy 'lzop -d'
>>
>> Perhaps you could first test whether these commands are available before
>> running try_decompress() with them?
>
> Same as before, if it's ok for extract-ikconfig not to check the
> command, then it's ok for me.
> If it's not, then we should patch both.
Also, if you have lzma to build a lzma-compressed kernel, then you very
likely also have unlzma.
Michal
Corentin Chary
> On 11.8.2011 15:08, Corentin Chary wrote:
>> On Thu, Aug 11, 2011 at 1:28 PM, Paul Bolle <peb...@tiscali.nl> wrote:
>>> On Thu, 2011-08-11 at 10:53 +0200, Corentin Chary wrote:
>>>> +try_decompress()
>>>> +{
>>>> + for pos in `tr "$1\n$2" "\n$2=" < "$img" | grep -abo "^$2"`
>>>> + do
>>>> + pos=${pos%%:*}
>>>> + tail -c+$pos "$img" | $3 > $tmp 2> /dev/null
>>>
>>> Perhaps a few comments on the above lines would be nice. Without those
>>> comments I must guess you're finding compressed data somewhere in the
>>> input file. It also seems you're looping through the entire input file.
>>> Or are (sequences of) commands like the above considered obvious?
>>
>> It's copied from extract-ikconfig, there was no comment in it, so I
>> assumed it's obvious.
>> Basically it uses brute force (tm) to find well known headers and try
>> to decompress from here.
>
> And explanation of the "tr" trick would not hurt though. It replaces the
> binary magic with an ascii sequence on a new line (for older grep
> versions to get it right) and tries all occurrences of that sequence, am
> I right? But I wouldn't call it "obvious" :).
It's that commented in the header of both scripts ? (I can move down
the command if wanted)
>
>>>> +# That didn't work, so retry after decompression.
>>>> +try_decompress '\037\213\010' xy gunzip
>>>> +try_decompress '\3757zXZ\000' abcde unxz
>>>> +try_decompress 'BZh' xy bunzip2
>>>> +try_decompress '\135\0\0\0' xxx unlzma
>>>> +try_decompress '\211\114\132' xy 'lzop -d'
>>>
>>> Perhaps you could first test whether these commands are available before
>>> running try_decompress() with them?
>>
>> Same as before, if it's ok for extract-ikconfig not to check the
>> command, then it's ok for me.
>> If it's not, then we should patch both.
>
> Also, if you have lzma to build a lzma-compressed kernel, then you very
> likely also have unlzma.
>
> Michal
>
--
Corentin Chary
http://xf.iksaif.net
Corentin Chary
> the command if wanted)
s/It's/Isn't/
Corentin Chary
kernel image (bzImage, etc..). It's inspired from (a subset of)
extract-ikconfig.
It's something a lot of people have been looking for (mainly
people with xen < 4 that doesn't support bzImages at all).
Signed-off-by: Corentin Chary <coren...@iksaif.net>
---
Since v1:
* update license
Since v2:
* add some comments
scripts/extract-vmlinux | 62 +++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 62 insertions(+), 0 deletions(-)
create mode 100755 scripts/extract-vmlinux
diff --git a/scripts/extract-vmlinux b/scripts/extract-vmlinux
new file mode 100755
index 0000000..4ab8517
--- /dev/null
+++ b/scripts/extract-vmlinux
@@ -0,0 +1,62 @@
+#!/bin/sh
+# ----------------------------------------------------------------------
+# extract-vmlinux - Extract uncompressed vmlinux from a kernel image
+#
+# Inspired from extract-ikconfig
+# (c) 2009,2010 Dick Streefland <di...@streefland.net>
+#
+# (c) 2011 Corentin Chary <corenti...@gmail.com>
+#
+# Licensed under the GNU General Public License, version 2 (GPLv2).
+# ----------------------------------------------------------------------
+
+check_vmlinux()
+{
+ # Use readelf to check if it's a valid ELF
+ # TODO: find a better to way to check that it's really vmlinux
+ # and not just an elf
+ readelf -h $1 > /dev/null 2>&1 || return 1
+
+ cat $1
+ exit 0
+}
+
+try_decompress()
+{
+ # The obscure use of the "tr" filter is to work around older versions of
+ # "grep" that report the byte offset of the line instead of the pattern.
+
+ # Try to find the header ($1) and decompress from here
+ for pos in `tr "$1\n$2" "\n$2=" < "$img" | grep -abo "^$2"`
+ do
+ pos=${pos%%:*}
+ tail -c+$pos "$img" | $3 > $tmp 2> /dev/null
+ check_vmlinux $tmp
+ done
+}
+
+# Check invocation:
+me=${0##*/}
+img=$1
+if [ $# -ne 1 -o ! -s "$img" ]
+then
+ echo "Usage: $me <kernel-image>" >&2
+ exit 2
+fi
+
+# Prepare temp files:
+tmp=$(mktemp /tmp/vmlinux-XXX)
+trap "rm -f $tmp" 0
+
+# Initial attempt for uncompressed images or objects:
+check_vmlinux $img
+
+# That didn't work, so retry after decompression.
+try_decompress '\037\213\010' xy gunzip
+try_decompress '\3757zXZ\000' abcde unxz
+try_decompress 'BZh' xy bunzip2
+try_decompress '\135\0\0\0' xxx unlzma
+try_decompress '\211\114\132' xy 'lzop -d'
+
+# Bail out:
+echo "$me: Cannot find vmlinux." >&2
Américo Wang
I don't know any way to check vmlinux, we can only check if it is ELF,
so this patch looks good.
Michal?
Corentin Chary
> On Tue, 2011-08-16 at 10:46 +0200, Corentin Chary wrote:
>> This script can be used to extract vmlinux from a compressed
>> kernel image (bzImage, etc..). It's inspired from (a subset of)
>> extract-ikconfig.
>
> the bzImage to find the payload rather than scanning (these are present
> in bzImages from somewhere in the mid 2.6.2x range). I'd be happy to
> license it under the GPLv2 if that is helpful.
I think it's good to know that this code exist, but it's better to
keep the current code that's both simple and stupid because I'd really
want to avoid implemeting a specific method for bzImages if the
generic one is solid enought. The current method has the great
advantage of being esasy to port to other formats.
>> It's something a lot of people have been looking for (mainly
>> people with xen < 4 that doesn't support bzImages at all).
>
> xen 3.4 does, doesn't it (at least the tip of 3.4-testing.hg does)? And
> I thought e.g. RHEL5 (which uses an older base version) had it
> backported. Possibly what is missing is support for all the various
> compression options.
Well I don't know about RHEL5, but I know that it doesn't work on
ubuntu 8.04 (and unfortunatly I'm stuck with one of these servers).
But this script can also be used for debuging purpose, or anything
else needing the raw vmlinux.
Thanks for the tips anyway :).
--
Corentin Chary
http://xf.iksaif.net
Corentin Chary
> Ping...
>
> I don't know any way to check vmlinux, we can only check if it is ELF,
> so this patch looks good.
>
> Michal?
Ping ?
--
Corentin Chary
http://xf.iksaif.net
Michal Marek
> This script can be used to extract vmlinux from a compressed
> kernel image (bzImage, etc..). It's inspired from (a subset of)
> extract-ikconfig.
>
> It's something a lot of people have been looking for (mainly
> people with xen < 4 that doesn't support bzImages at all).
>
> Signed-off-by: Corentin Chary <coren...@iksaif.net>
> ---
>
> Since v1:
> * update license
>
> Since v2:
> * add some comments
>
> scripts/extract-vmlinux | 62 +++++++++++++++++++++++++++++++++++++++++++++++
Applied to kbuild-2.6.git#misc, thanks.
Michal