[Patch suggestion] Booting Android-x86 via PXE/NFS

[Gernot Hillier]

Hi!

For faster development cycles, I prefer netbooting an NFS-mounted root
filesystem instead of booting flash or ISO images.

After some fiddling, I got this to work with android-x86 on an Atom
panel PC. Thus, I can immediately boot the target from the build
directory on the host w/o the need to write any CD/USB media.

Basically, I booted via PXE (which is available in most x86 systems'
BIOSes today) and NFS into the installer/ root so that things go the
same way as during a CD boot.

This required a minimal patch to bootable/newinstaller/initrd/init which
you can find attached. If someone wants to test it, please replace the
nfsroot=... parameter reflecting your setup.

So... What do you think about integrating this feature into mainline?

Certainly, the hard-coded NFS path in the attached patch is not for
mainline integration. But before polishing this (I thought
about shell code to extract the actual NFS root from /proc/cmdline),
I wanted to hear your general opinion.

A short howto will follow for experienced early adopters ;) - if you
plan for mainline integration, I can surely provide a more detailed
howto for your homepage.

--------

Set up a TFTP, DHCP and NFS server on the host. The NFS server exports
out/target/product/generic_x86/installer/ from the android-x86 build
directory.

For PXE, I provided pxelinux.0 from the OpenSUSE syslinux package to the
TFTP root (usually /srv/tftpboot). Create <TFTProot>/pxelinux.cfg/default,
holding the boot config:

------ SNIP -----
prompt 1
timeout 50
default android

label android
kernel kernel
append ip=dhcp root=/dev/nfs rw nfsroot=192.168.0.100:/path/to/android-x86/out/target/product/generic_x86/installer/ console=ttyS4,115200 DEBUG=1 initrd=initrd.img
------ SNIP -----

The "append" line can be extended using parameters from out/target/
product/generic_x86/boot/isolinux/isolinux.cfg.

Don't forget to replace the nfsroot path with your actual NFS export.
The console=... setting needs to be adapted to your serial port
settings or removed if you want to use the local console.

--
With kind regards,

Gernot Hillier
Siemens AG, CT T, Program "Open Source Platforms"

[fuzzy7k]

I like the idea. I had set up something similar a while back but there was not much interest in it. I think mainly because nfs support was not built into busybox, nor was there interest in adding it. The way it was explained to me is you could netboot it by transferring the entire iso as the ramdisk. I never did try it that way.

One suggestion, there is no need to parse /proc/cmdline. Just use your variable, $nfsroot.

[Gernot Hillier]

Hi!

Am 17.04.2012 13:07, schrieb fuzzy7k:
> I like the idea. I had set up something similar a while back but there
> was not much interest in it. I think mainly because nfs support was not
> built into busybox, nor was there interest in adding it.

It worked without any busybox modifications. There are only two caveats:

- I needed to mount with "-o nolock" to avoid necessity of portmap,
lockd and friends.
- I needed to add a small workaround to the installer shell function
remount_rw (just use an arbitrary mount source, it's ignored anyways)
as Busybox seems to stumble while parsing nfs-specific mount options
when the mount source isn't mentioned explicitely.

Including those workarounds, it was really only about changing three
lines of source code.

> The way it was explained to me is you could netboot it by
> transferring the entire iso as the ramdisk. I never did try it that
> way.

Yeah, I also found that when starting my work - however, I wanted to
speed things up - and doing a full TFTP ISO download on each reboot
didn't seem the right way to me. (Not mentioning, that I don't have to
wait for mkisofs to finish before I can boot up my target after a change.)

And, booting without ISO could also help speeding up development cycles
for emulator boot, as a colleague recently mentioned. However, I didn't
try this yet.

> One suggestion, there is no need to parse /proc/cmdline. Just use your
> variable, $nfsroot.

Ah, great, thanks. Didn't think of that... :)

--

[Chih-Wei Huang]

2012/4/17 Gernot Hillier <gernot....@siemens.com>:

> Hi!
>
> Am 17.04.2012 13:07, schrieb fuzzy7k:
>
>> I like the idea. I had set up something similar a while back but there
>> was not much interest in it. I think mainly because nfs support was not
>> built into busybox, nor was there interest in adding it.
>
> It worked without any busybox modifications. There are only two caveats:
>
> - I needed to mount with "-o nolock" to avoid necessity of portmap,
>  lockd and friends.
> - I needed to add a small workaround to the installer shell function
>  remount_rw (just use an arbitrary mount source, it's ignored anyways)
>  as Busybox seems to stumble while parsing nfs-specific mount options
>  when the mount source isn't mentioned explicitely.
>
> Including those workarounds, it was really only about changing three lines
> of source code.

Sorry for reply late.

I think the -o nolock and remount_rw changes are OK.
However, hardcoded the nfsroot in the script seems not good.

+ for device in ${ROOT:-/dev/sr* /dev/[hs]d[a-z]*
192.168.0.100:/media/gernot-projects/android-x86/out/target/product/generic_x86/};
do

Could you just set ROOT=192.168.0.100...
in your cmdline? It should work. Shouldn't it?

>> The way it was explained to me is you could netboot it by
>> transferring the entire iso as the ramdisk. I never did try it that
>> way.
>
>
> Yeah, I also found that when starting my work - however, I wanted to speed
> things up - and doing a full TFTP ISO download on each reboot didn't seem
> the right way to me. (Not mentioning, that I don't have to wait for mkisofs
> to finish before I can boot up my target after a change.)
>
> And, booting without ISO could also help speeding up development cycles for
> emulator boot, as a colleague recently mentioned. However, I didn't try this
> yet.
>
>
>> One suggestion, there is no need to parse /proc/cmdline. Just use your
>> variable, $nfsroot.
>
>
> Ah, great, thanks. Didn't think of that... :)

--
Chih-Wei
Android-x86 project
http://www.android-x86.org

[Gernot Hillier]

Hi!

On 2012-04-25, Chih-Wei Huang wrote:
> 2012/4/17 Gernot Hillier<gernot....@siemens.com>:

>
>> It worked without any busybox modifications. There are only two caveats:
>>
>> - I needed to mount with "-o nolock" to avoid necessity of portmap,
>> lockd and friends.
>> - I needed to add a small workaround to the installer shell function
>> remount_rw (just use an arbitrary mount source, it's ignored anyways)
>> as Busybox seems to stumble while parsing nfs-specific mount options
>> when the mount source isn't mentioned explicitely.
>>
>> Including those workarounds, it was really only about changing three lines
>> of source code.
>
> Sorry for reply late.

No problem, we all have other tasks sometimes. ;-)

> I think the -o nolock and remount_rw changes are OK.

Probably it makes sense to only use "-o nolock" in case of NFS mounts. I
updated the patch, see attachment. The remount_rw changes should never
harm, so I left them unchanged.

> However, hardcoded the nfsroot in the script seems not good.

I can deeply confirm this since I recently lost some debugging hours
because booting from an outdated server share due to the hardcoded
string in the script. :-)

However, this part was never meant to end-up in your mainline, just to
demonstrate how things are working here.

> + for device in ${ROOT:-/dev/sr* /dev/[hs]d[a-z]*
> 192.168.0.100:/media/gernot-projects/android-x86/out/target/product/generic_x86/};
> do
>
> Could you just set ROOT=192.168.0.100...
> in your cmdline? It should work. Shouldn't it?

... I planned to grep for nfsroot= in /proc/cmdline, but you're totally
right - setting ROOT=... from the bootloader seems by far the best
solution. Hunk removed.

So could you please apply the attached patch? If you prefer submission
in another form or pulling from some git branch, please let me know so I
can update it.

And please let me know in which form I could submit a short howto,
probably to be included on android-x86.org?

>>> One suggestion, there is no need to parse /proc/cmdline. Just use your
>>> variable, $nfsroot.
>>
>> Ah, great, thanks. Didn't think of that... :)

BTW, @fuzzy7k: using $nfsroot doesn't work, as it's an option understood
by the kernel and thus not added to userspace environment.

--
Gernot

[Chih-Wei Huang]

Thank you! The patch is applied.

2012/4/27 Gernot Hillier <gernot....@siemens.com>:

[Pedro Ferreira]

Hello all,
A couple of questions:

1. Have you made a howto on how to use android and pxe?

2. Is there a way of booting android live without the NFS partition? I tried this on my pxelinux.cfg/default configuration file and it loads kernel and system.sqs but I have a kernel panic at the end saying it can't mount the root file system:
label 4
  kernel AndroidLive03/kernel
  append rw initrd=AndroidLive03/system.sqs ip=dhcp

greetings
Pedro

[fuzzy7k]

Yes, instead of modules, you have to build into the kernel the driver for your network card and nfs file system support.

On Wednesday, March 20, 2013 1:12:07 PM UTC-4, kwu wrote:

Will the contents of the default .iso file work with pxe? Does the kernel have to be reconfigured/recompiled to enable nfs mounting?

[kwu]

Is there any changes to networking between the latest jb source and ics? I am using VirtualBox as the guest to boot from a NFS location. When I enabled NFS client and NFS root support through menuconfig on the ics source it worked perfectly. However when I did the same on the jb source, it connected to the TFTP but never connect to the NFS server. Is there some difference between those two sources that I'm missing?

[kwu]

I found it. The Intel(R) PRO/1000 Gigabit Ethernet driver under Network device support -> Ethernet driver support was not included in jb. Enabling it made it work. Thanks.

[Chih-Wei Huang]

Hmm, did you mean CONFIG_E1000?
I think it is enabled in jb-x86.
You can check kernel/arch/x86/configs/android-x86_defconfig

CONFIG_E1000=m
CONFIG_E1000E=m
...

2013/4/1 kwu <quantu...@gmail.com>:

> I found it. The Intel(R) PRO/1000 Gigabit Ethernet driver under Network
> device support -> Ethernet driver support was not included in jb. Enabling
> it made it work. Thanks.
>
>
> On Sunday, March 31, 2013 11:40:59 PM UTC-5, kwu wrote:
>>
>> Is there any changes to networking between the latest jb source and ics? I
>> am using VirtualBox as the guest to boot from a NFS location. When I enabled
>> NFS client and NFS root support through menuconfig on the ics source it
>> worked perfectly. However when I did the same on the jb source, it connected
>> to the TFTP but never connect to the NFS server. Is there some difference
>> between those two sources that I'm missing?

[kwu]

Yeah it was enabled as a module <m> in menuconfig as well. But it didn't load the driver until I included it. =y. In the ICS source, CONFIG_E1000=y so I didn't need to do any extra steps besides enabling NFS_ROOT.

[fuzzy7k]

... and NFS_ROOT is not a requirement for the way this is implemented.

[Pccom Frank]

Hi Chih-Wei Huang!

Have you checked the nfsroot works with the latest Android lollipop?

I have tested several times and never made it work.

First, the /system/xbin/mount -t nfs -o nolock xxx.xxx.xxx.xxx:/opt/nfs/android-2015-02-08/ /mnt/obb won't work in the debug mode until I add some module in the kernel to enable NFSv4 and NFSv4.1.

Before I do mount, I have to modprobe e1000, sunrpc, lockd and nfs, Then I have to ifconfig eth0 192.168.0.111 netmask 255.255.255.0

route add default gw 192.168.0.1

After I made userland debug mode nfs working, I was wishing it would work in the kernel if I feed the command line with root=/dev/nfs ip=dhcp nfsroot=<the nfsroot file system at the nfs server> ROOT=<the nfsroot file system at the nfs server>

The result is 

Booting the kernel ..... forever

It seems the kernel never made a nfs connection to the nfs server.

I served both initrd.img and kernel from my ipxe server.

It seems the kernel loaded the initrd.img into memory but the init program can never find the nfs mount.

I was thinking it is easier for the kernel to find the /dev/nbd0 device than to mount a nfs file system. Then I enabled nbd in the kernel. Then in the debug mode, I can never make the nbd-client to work. I guess the busybox nbd-client not working. But there is no way to enable busybox to compile nbd-client. Are you using a binary busybox? How can I select busybox to compile nbd-client? Is there a make -C busybox TARGET_PRODUCT=android_x86 menuconfig?

Can you give me some hint to get NFS or NBD PXE boot android_x86 ?

Thanks!

Frank Peng

[Chih-Wei Huang]

Hi,
I didn't test the NFS mounting feature myself.
If you think we missed some kernel config to support it,
send me a patch.
Thank you!

> --
> You received this message because you are subscribed to the Google Groups "Android-x86" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to android-x86...@googlegroups.com.
> To post to this group, send email to andro...@googlegroups.com.
> Visit this group at http://groups.google.com/group/android-x86.
> For more options, visit https://groups.google.com/d/optout.

[fuzzy7k]

The main problem with netboot support is including the drivers in initrd. This is not done. The way that I did it was to compile a custom kernel, and use gpxe to setup up the network, and download the files. I only used gpxe, because it allowed me to add a boot option in the grub menu, instead of going through the BIOS.

The other problem is that the boot process is a little outside the box. There are two ramdisks for example. If I remember correctly, this was why setting root=/dev/nfs did not work, because it needed to be /dev/ram0 for boot to work.

[Pccom Frank]

I am still detecting android_x86 ................. forever.

What I did:

I re-compiled kernel with NFS 4.1 support. Then,

In the debug mode(busybox shell), I did following:

modprobe e1000

modprobe sunrpc

modprobe lockd

modprobe nfs

ifconfig eth0 192.168.0.111 netmask 255.255.255.0

route add default gw 192.168.0.1

mkdir /mnt/nfs

/system/xbin/mount -t nfs -o nolock 192.168.0.1:/opt/nfs/android-2015-02-08 /mnt/nfs

It worked. 

But I do know how to switch_root to /mnt/nfs. I have no way to enter the /sbin/init program to start OS.

Then I go back to the kernel, I found all modules above are m. 

The second question is that all the modules needed by nfs are loaded in the kernel or not? The load_modules function worked or not? I do not know.

This is the init program:

......

 # load scripts

for s in `ls /scripts/* /src/scripts/*`; do

        test -e "$s" && source $s

done

# A target should provide its detect_hardware function.

# On success, return 0 with the following values set.

# return 1 if it wants to use auto_detect

......

Third question is that "Do we have a script for nfs detecting?"

After all of these success, it will go to the /sbin/init program, which is using the init.rc configuration file. 

I saw in the scripts/0-auto-detect program, it is shooting the init.rc file.

After nfsroot every success, if we cannot write init.rc right, still it will not work:

Third question is, is that possible that the NFS filesystem worked, but it won't pass to operating system, that means the binary file /sbin/init won't work with the configuration file: init.rc.

.........

find_network_dev_name()

{

        INITRC=init.x86.rc

        DHCPCD="/system/bin/dhcpcd -o domain_name_servers"

        rmline=`grep -n "#REMOVE FROM HERE" $INITRC | cut -d':' -f1`

        rmline=`expr $rmline + 1`

        sed -i -e "$rmline,\$d" $INITRC

        for netdev in `ls /sys/class/net`; do

                case "$netdev" in

                        lo|ip6tnl*|sit*|tun*|eth0|wlan0|wmaster*)

                                ;;

                        *)

                                echo -e "\nservice dhcpcd_$netdev $DHCPCD -ABDKL\n    class main\n    disabled\n    oneshot\n" >> $INITRC

                                echo -e "service iprenew_$netdev $DHCPCD -n\n    class main\n    disabled\n    oneshot" >> $INITRC

                                ;;

                esac

        done

}

load_modules()

{

        if [ -n "$FOUND" ]; then

                [ -n "$PREDEV" ] && modprobe $PREDEV

                [ -n "$FB0DEV" -a -z "$UVESA_MODE" ] && modprobe $FB0DEV

                [ -n "$LANDEV" ] && modprobe $LANDEV

                [ -n "$WIFDEV" ] && modprobe $WIFDEV

                [ -n "$SNDDEV" ] && modprobe $SNDDEV

                [ -n "$CAMDEV" ] && modprobe $CAMDEV

        else

                auto_detect

        fi

        # 3G modules

        EXTMOD="$EXTMOD pppoe ppp-async ppp-deflate bsd-comp"

        for m in $EXTMOD; do

                modprobe $m

        done

        load_uvesafb

        lsusb -v | awk ' /Class:.E0/ { printf("%s      0660   bluetooth bluetooth\n", $9) } ' >> ueventd.android_x86.rc

}

.....

These are all dynamic device handling functions. It is extremely hard to understand and modify.

If you can give any hint for help, I will appreciate it!

Frank Peng

--
You received this message because you are subscribed to a topic in the Google Groups "Android-x86" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/android-x86/SdaOXm1Mt0g/unsubscribe.
To unsubscribe from this group and all its topics, send an email to android-x86...@googlegroups.com.

[Chih-Wei Huang]

2015/2/12 下午1:50 於 "Pccom Frank" <pccom...@gmail.com> 寫道：
>
> I am still detecting android_x86 ................. forever.
>
>
> What I did:
>
> I re-compiled kernel with NFS 4.1 support. Then,
>
> In the debug mode(busybox shell), I did following:
>
> modprobe e1000
> modprobe sunrpc
> modprobe lockd
> modprobe nfs
>
>
> ifconfig eth0 192.168.0.111 netmask 255.255.255.0
> route add default gw 192.168.0.1
> mkdir /mnt/nfs
> /system/xbin/mount -t nfs -o nolock 192.168.0.1:/opt/nfs/android-2015-02-08 /mnt/nfs
>
> It worked. 
>
> But I do know how to switch_root to /mnt/nfs. I have no way to enter the /sbin/init program to start OS.

You meant you do NOT know? Why?
Switching root to nfs is nothing different than switching to others.

Depends on how many stuff you hope to put on nfs,
there are two approaches (at least):

* All stuff are on nfs including the first init process
  In this case you need the nfsroot support.
  You need deeper knowledge about linux and android booting.
  More difficult but still doable.

* You can load initrd.img locally so you already have the basic
  filesystem, then try to mount nfs and switch to it.

From your description I think you tried the second approach.
This should already be supported by the ROOT variable of initrd.img.
Just specify this in cmdline:

ROOT=192.168.0.1:/opt/nfs/android-2015-02-08

You just need to make sure the nfs contains the
ramdisk.img and system.sfs (or system.img).

> Then I go back to the kernel, I found all modules above are m. 

That's an issue.
The simplest solution is just change them to y.
Or you need to put them into initrd.img
and write script to load them.

> The second question is that all the modules needed by nfs are loaded in the kernel or not? The load_modules function worked or not? I do not know.
>
> This is the init program
> ......
>  # load scripts
> for s in `ls /scripts/* /src/scripts/*`; do
>         test -e "$s" && source $s
> done

This allows you to hook customized scripts
without rebuilding the initrd.img.
But it may not be helpful to your problem...

> # A target should provide its detect_hardware function.
> # On success, return 0 with the following values set.
> # return 1 if it wants to use auto_detect
> ......
>
> Third question is that "Do we have a script for nfs detecting?"

What do you want to detect exactly?
If you want a script to load necessary modules, apparently no.

> After all of these success, it will go to the /sbin/init program, which is using the init.rc configuration file. 
>
> I saw in the scripts/0-auto-detect program, it is shooting the init.rc file.
> After nfsroot every success, if we cannot write init.rc right, still it will not work:
> Third question is, is that possible that the NFS filesystem worked, but it won't pass to operating system, that means the binary file /sbin/init won't work with the configuration file: init.rc.

These functions are considered deprecated and may be removed in the future.
The current android-x86 implementation doesn't rely on them at all.
So you don't need to bother with them.

You received this message because you are subscribed to the Google Groups "Android-x86" group.

To unsubscribe from this group and stop receiving emails from it, send an email to android-x86...@googlegroups.com.

[Ye Roger]

This is very useful message. I am trying to do the same using VirtualBox. I can do pxe boot using VirtualBox to load kernel and ramdisk now. I am just wondering how should I create the NFS root. As you know the root of Android is ramdisk actually, it mount system/ and data/ to ramdisk and boot the android system.

My question here is:

1. Do we still use ramdisk as root file system for NFS boot ? If yes, we need to mount system/ and data/ to it using NFS.
2. If not, we cannot use out/target/product/x86/installer as root, since it is quite empty. It is suppose to be the root for ISO boot for installation.
3. What in my mind, the NFS root should be something like ramdisk with system/ and data/ folders as a live Android file system. Is this a right assumption ? What's your NFS root configuration ?
   

-Roger

[a.zhem...@gmail.com]

Hi Chih-Wei,

I'm sorry to bother,but I would you to help me with my situation.THANK YOU!

I'm trying to boot Android-x86 via NFS with wireless.So I need to configure wireless network when the system is booting while not it is booted.

Now,I put my commands in script 'init' in initrd.img,and I also add and insert related modules(wireless).

I use wpa_supplciant when I connect to a router.

It just can't work!

Do I miss something?

Sorry for my poor English and it's my first time.

Any help will be appreciated!  

[Hassin Ayaz]

Hello , Did you succeeded over Ethernet ? 

could you kindly tell how you did it ... 

i have already posted for help in this post https://groups.google.com/d/msg/android-x86/pGcPFOBwCkc/_kG1fk4xAwAJ where i described my steps , many of the tinkering were applied following this thread's information 

Thanks