Raspberry PI boot to network storage with Berryboot

[Luc Heijst]

Berryboot is a simple boot selection screen, allowing you to put multiple Linux distribution on a single SD card.

In addition it allows you to put the operating system files on an external USB hard drive or on a networked storage instead of on the SD card itself.

In the latter two cases the SD card is only used during boot time which reduces the chance of a broken card.

See description and installation instructions for Berryboot: http://www.berryterminal.com/doku.php/berryboot

For my Raspberry PI model 2B I selected a network storage on my Synology NAS instead of my USB-stick I used before.

As a result the network transmit on the RPI is increased and the cpu_iowait now is practically zero. See attached graphs.

Also a screen shot of the Storage Manager and Resourse Monitor of my Synology NAS

Luc

[vince]

On Tuesday, April 19, 2016 at 11:22:49 AM UTC-7, Luc Heijst wrote:

Berryboot is a simple boot selection screen, allowing you to put multiple Linux distribution on a single SD card.

In addition it allows you to put the operating system files on an external USB hard drive or on a networked storage instead of on the SD card itself.

Pretty cool...

I looked at their web page and the following concerned me:

Be aware that if you choose an external drive, the files of the operating system will be stored there, but you still need to keep the SD card in the Pi to boot from.

WARNING: all existing files on the disk will be erased.

I don't have a Synology but I do have a little debian box with some free disk space.  Looking at http://www.server-world.info/en/note?os=Debian_8&p=iscsi it 'seems' like I could create a disk image file there with dd  and serve it up via iSCSI as a fixed-size target for the pi(s), but I want to make sure that berryboot doesn't reformat my whole box of course. Anybody have any success doing that ?

[Luc Heijst]

Vince,

You have to create an iSCSI LUN (with regular files) on your NAS or computer system with a fixed size (I choosed 8 GB, the size can be altered later). 

Then map on your NAS or computer system an iSCSI target to that lun. The iSCSI target has an own iqn (e.g. iqn.2000-01.com.synology:<system name>.Target-xxxxxxxxxxxx).

In the Berryboot menu you can search for all iSCSI targets on the selected system (ip-address) and select one.

When configured, Berryboot will format the space on the iSCSI target (in our example the 8 GB iSCSI LUN).

It is possible to assign a whole disk to a iSCSI LUN (block-level single LUN on disk), but in most cases you don't want that. On my Synology NAS this option was greyed out, because the disk was not empty.

Hope this helps.

Luc

[Luc Heijst]

Correction: 

I don't think you can search for all iSCSI targets in the Berryboot menu. 

For more information on a similar subject look at: https://www.reddit.com/r/raspberry_pi/comments/49l25b/iscsi_booting_your_raspberry_pi/

Note: I did not succeed to get this working, so I switched (back) to Berryboot with the knowledge I learned in the other blog.

Luc

[Luc Heijst]

You could look at Berryboot's source code of course, then you would know for sure! :-) 

https://github.com/maxnet/berryboot

[Glenn McKechnie]

On Wednesday, 20 April 2016 07:44:25 UTC+10, Luc Heijst wrote:

You could look at Berryboot's source code of course, then you would know for sure! :-) 

https://github.com/maxnet/berryboot

Ah, Buildroot. I keep coming across that in my search for alternative systems - ones that minimize SDcard writes by mounting / as read-only.

I did try buildroot, rpi-buildroot in fact, but because I was using (as it now turns out) two dodgy SDCard readers (the3rd one was the charm), I failed to get a succesful installation. Very off-putting but I'm back on track now.

Pushing files across the network... You're using this - berryboot - with your mysql databases? In particular a busy one that is being updated constantly, such as Mesowx being fed by a Davis weather station - or owfs setup if it was mine. I'm thinking of heavy traffic where the updates are less than, say, 10 seconds apart. Do those stats you show reflect that traffic?
Obviously this is not over a wireless network. I'd expect the answer to be a definite No! for that, unless my wildest dreams have been answered.
What OS's have you installed under berryboot and what kernels are booting those, I'm assuming the berryboot kernel? No mismatch in modules?

Currently I'm trying voyage linux - mubox http://mubox.voyage.hk/ (on my flaky connection the site's down, but google cache confirms the link ). That shows a media player, but the source repository is available and the image does install (with a few tweaks) on either a B+ or a Pi 2 B (auto-detect on bootup). Admittedly it does need a couple of edits which I've noted on their ultra-quiet mailing list.
Some fiddling is required to repurpose it to a weewx setup but it runs Debian so updating it is easy, and is what attracted me to it in the first place.  My installation is running weewx with a WS108 and 4 months of imported / backlogged data just fine - a few issues with mysql given the database size and available memory on the Pi 2B. Weewx's running with sqlite now and the pi has had no memory issues at all - it is a finite resource after all.
 
If there is interest here and anyone want the details just ask and I'll clean up my notes. To recap / (root) is readonly (except for software installations / updates as required) tmpfs is used for the weewx database, the html directory and of course logs. If you choose to, then a couple of config files can be thrown into the write portion as well - skins, configs, whatever - you just can't get greedy.

As for writes to the card, after the initial installation and setup there should be none - ever, it's running from memory. As of course does berryboot and others.  For multiple installations, once one is configured then that SDcard can be copied back to the Deskop machine as an installable image for reuse elsewhere. While writes will still be required on the next card, the major work of converting to a current version, as well as running weewx will have been done.

There's nothing stopping the read write filesystem being mounted as an actual disk rather than residing in memory and that would reduce any 'out of memory', 'Failed to fork'  problems (I'm pointng my finger at apt-get there); and not use up a rare USB slot (if on a B+).
 
And, getting back on topic, you've raised my interest in the network traffic involved. If that was minimal, or not an issue, then the read/write filesystem can be located offsite and yet another solution arises.

[L.J.M. Heijst]

Glenn,

My RPI 's and NAS'es all are part of a local gigabit cabled network (of course!).

Running three instances of weewx on one RPI2 (RPI3 is expected in two days).

The NAS is a DS211 with gigabit network port.

I have 5 remote weewx MySql databases active (now they happen to be on the same NAS as the debian OS files, but still configured as remote so that Synology can make the daily database backups.

I have also 2 remote MySql databases running on my webserver in the Netherlands (7000 km distance!) They receive online loop data (2,5 s interval) via two separate sync calls for MesoWX and Meteotemplate respectively and of course the 5-minute data and html stuff. The real bottleneck in the system is (and was) FTP'ing the 90+ graph files each archive interval to the web server. The average of one file per second is now a bit worse. The cpu and memory of my NAS are half as good as of the RPI2 but I don't think that matters in this configuration.

I have installed via Berryboot debian jessie light (2016-03). The SD card has a berryboot kernel, but also two RPI kernels: one for PI0 and PI1 and one for PI2 and PI3. No mismatch so far. I did a apt-get upgrade on the fresh installed jessie.

To test my configuration a bit further I also performed two firmware upgrades on my NAS while everything was running. I had to restart the weewx instances after the upgrade. Not sure if the automatic start of weewx after the PI boot still work. Maybe I'll have to wait longer for the network to establish.

Luc

Verstuurd vanaf mijn iPad

--
You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/3Dg1l67DpRY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Luc Heijst]

PS. My weewx databases on the Synology NAS are in fact MariaDB databases. Don't know if that matters in respect to performance.
Luc

[Glenn McKechnie]

You work them hard Luc :-)

Thanks for the info, it completes the picture.

It won't work as well here as it's a mix of 100M cable and wireless. I used etherboot (TLSP) for a Dachstein (LRP) router once and it worked seamlessly but that was a small, single transfer and the tftpboot was done with. You've still got me thinking though how or if it could be worked into the final set up here .
Regarding the 90 + files. I'm doing the math and assuming these are for the weewx skins. It's easy to take for granted just what a single weewx installation does for one skin, let alone several instances with added extras.

cheers
Glenn

[Luc Heijst]

Hi Glenn,

So much to do, so little time left... (just got 65 and a bit joking!)

The remote OS solution left me with one spare USB port. That would be a waste not to use!

So, since yesterday I hooked up my brand new meteostick to this USB port.

At first there was a conflict with the Vantage Pro2 console and the meteostick which both could not decide if they should be connected to /dev/ttyusb0 or /dev/ttyusb1.

So I followed the reccomendations in the weewx documentation (thanksTom!) to set up two udev devices: /dev/vpro and /dev/'mstk.

Now they are happilly working together without complaining!

If you promise not to laugh I will reveil my current hardware setup (see attachment)

Cheers,

Luc

[Thomas Keffer]

I'm sure that roll of toilet paper figures in it.

-tk

--
You received this message because you are subscribed to the Google Groups "weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to weewx-user+...@googlegroups.com.

[L.J.M. Heijst]

Tom,

• It's cheep
• It was availabe
• It is reusable
• It won't disturb radio signals
• It's soft to your (well, never mind)

Luc

. -- You received this message because you are subscribed to a topic in the Google Groups "weewx-user" group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/weewx-user/3Dg1l67DpRY/unsubscribe. To unsubscribe from this group and all its topics, send an email to weewx-user+...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.

Virusvrij. www.avast.com

[Geoff Cusick]

A quick question:

I've got a working weewx system on an RPi (www.weather.cusick.org.uk), which I'd like to change so that it runs completely on a NAS on my network. I've used Berryboot, and have a test system running Raspbian from an iSCSI partition, so that bit works. My question is: can I take an image of my weewx system and install it complete in Berryboot, or do I need to rebuild the weewx system in situ?

Any experience of this?

Thanks
Geoff

[Luc Heijst]

Geoff,

What works for me (not necessary the best method) is to copy the complete /home/weewx branch to the new system and then do an "update" with the newest weewx version with

python setup.py build

python setup.py install

You also need to install the required packages for weewx by:

apt-get install python-configobj  python-cheetah  python-imaging python-serial python-usb python-mysqldb python-dev python-pip rsync -y 

pip install pyephem

When you use mysql or mariadb as database you need to perform a backup/restore as well.

Luc  

[Geoff Cusick]

Thanks Luc - as always helpful advice.

I'm about to try creating a squashfs version of a complete Raspbian image with weewx etc installed, and adding that to Berryboot as a custom OS.  If that works, fine, I'm home and dry.  If not, I'll follow up your suggestion.

Regards
Geoff