Can't seem to get rid of stupid problems!

61 views
Skip to first unread message

Rilhas

unread,
Mar 18, 2012, 3:39:31 PM3/18/12
to Bifferboard
Hi all,

My "rilhas.com" Bifferboard is down again, with another case of
"BIFFDISK has been mounted 25 times without being checked, check
forced." and then "BIFFDISK: Error allocating icount link information:
Memory allocation failed e2fsck: aborted".

The "/fastboot" file is there, but as it seems it is being ignored.

I understand if my Bifferboard is unable to boot because something is
irrecoverably failing, but I am completely unable to understand why it
should fail when everything is alright "but, just in case, lets check
and then irrecoverably fail because we can't check".

Does anyone have a permanent and definitive solution for this (and
similar yet unknown to me) problems?

Thanks!

Fabio Silva

unread,
Mar 18, 2012, 5:13:51 PM3/18/12
to biffe...@googlegroups.com
My immediate suggestion for that particular problem would be to take the pen/disk off the bifferboard, and run the fsck on it while having it connected to a PC or VM with Linux installed.

Can you remind me of what is the Filesystem you are using for the pen/disk? Ext3? Ext4?

The problem here is that when the Linux init sequence is trying to FSCK the partition in question, it runs out of memory (can't use the swap file yet). How big is the partition?

My other suggestion would be to build a boot partition with a small size (2GB, perhaps) and a data partition with the remainder of the disk and on the fstab, set it to mount only on pass2 when you have the swap file already enabled so that you don't run out of memory.

Hope this helps.
Fabio

> --
> To unsubscribe send email to bifferboard...@googlegroups.com

Rilhas

unread,
Mar 18, 2012, 5:35:09 PM3/18/12
to Bifferboard
This was already discussed in one of my previous posts a few weeks
back, sorry I didn't remember to establish the relation in this new
post.

The Bifferboard is no longer running from a flash disk. It now has a
500GB hard disk. All went well until after 25 days (at one reboot per
night it took 25 nights to force a check).

One of the sugestions was that I added /fastboot, which I did. It
seemed to fix it, so I was ready to delay the real fix - splitting the
disk into a smaller boot partition and swap and a large partition with
all the rest (or a variation where the swap would have its own
partition).

However, the /fastboot doesn't seem to have fixed it at all, because
today I had the same problem despite the fact the /fastboot file is
there.


> My immediate suggestion for that particular problem would be to take the pen/disk off the bifferboard, and run the fsck on it while having it connected to a PC or VM with Linux installed.

Already doing that, because after it requests a filesystem check I can
find no other way to calm the beast except performing a filesystem
check on the PC. This typically takes several hours, so I have more
than enough time to wait for comments to my posts! :-)

> Can you remind me of what is the Filesystem you are using for the pen/disk? Ext3? Ext4?

Ext3. Ubuntu says Ext3 version 1.0 in the disk utility. Why not Ext4?
I wish I had an answer to that, but I don't. You were the one who
installed it for me, remember? You don't? Well, you did. Why did you
choose Ext3? I don't know but we talked about using Ext4, so my best
guess is that you probably found some reason why that wouldn't work
and decided to use Ext3 instead.

>
> The problem here is that when the Linux init sequence is trying to FSCK the partition in question, it runs out of memory (can't use the swap file yet). How big is the partition?

Yup. That is the problem. And that is what I call a stupid problem. I
would never have guessed that I file checking application would need
an amount of memory proportion to the size of the filesystem it
checks. And even worse: I could never have imagined that Linux would
find a way not to boot even if everything is ok. But I'm trying not to
be pulled to the dark side of the force because of this, so I will
keep my cool.

The partition is 500GB.

>
> My other suggestion would be to build a boot partition with a small size (2GB, perhaps) and a data partition with the remainder of the disk and on the fstab, set it to mount only on pass2 when you have the swap file already enabled so that you don't run out of memory.

Yes. In my previous post I mentioned I would set a reminder on my
phone to do that in about 6 months from then. It's a mental trick I
use to convince myself that I will actually do it, when, in practice,
when the time comes I guess I will probably just postpone the reminder
another 6 months. I use reminders as a way to don't forget to
procrastinate.

So the definitive solution for me, at this point, is all about
skipping the disk checks. That was the point of the /fastboot. I now
read that if I go to fstab and edit the last 1 to a 0 on the line
corresponding to the root partition then I may get away without forced
checks. But I imagine I still have to wait a couple of hours for the
disk check to complete before I can test this, so any comments are
welcome.

Fabio Silva

unread,
Mar 18, 2012, 6:02:49 PM3/18/12
to biffe...@googlegroups.com
You can also configure fsck to use a temporary file instead of using memory:

Create /etc/e2fsck.conf with the following contents:

[scratch_files]
directory = /var/cache/e2fsck

Rilhas

unread,
Mar 18, 2012, 7:07:25 PM3/18/12
to Bifferboard
The fstab thing worked.

... unless you hear from me again next month! :-)

Andrew Scheller

unread,
Mar 18, 2012, 11:57:29 PM3/18/12
to biffe...@googlegroups.com
> You can also configure fsck to use a temporary file instead of using memory:
> Create /etc/e2fsck.conf with the following contents:
> [scratch_files]
> directory = /var/cache/e2fsck

Wouldn't that only work if /var was on a different partition to the
one being fsck-ed?

>> So the definitive solution for me, at this point, is all about
>> skipping the disk checks. That was the point of the /fastboot. I now

http://lmgtfy.com/?q=linux+disable+automatic+fsck

;-)

Lurch

Rilhas

unread,
Mar 19, 2012, 9:02:56 AM3/19/12
to Bifferboard
>
> http://lmgtfy.com/?q=linux+disable+automatic+fsck
>
> ;-)
>
> Lurch

You make that seem soooo easy! :-)

One of the biggest problems I have with Linux is finding the correct
terminology for things. Information seems to be spread all over the
place and using a less-than-optimal search word does not take me to
the solution, but not right away, just after it takes me on a
sightseeing tour to teach me that there are other problems out there
that I, some day, will eventually meet, but only after I forget how to
find solutions for them. Also, it usually takes soooo long to read
through posts to see what people have been trying and, of those, which
have succeeded and, of those, which found a solution that is
applicable to my case, and of those go back and see if their problem
was actually the same as mine, that I usually prefer to just ask the
question here! At least the context for all of us seems to be the
same, as we are interested to get solutions for the Bifferboard your
answers here are already a big step forward getting a solution!

In this case I eventually came to the fstab solution, but not before
searching for various combinations of "linux disable forced filesystem
check". Many pointed to a "grub.conf" thing that I couldn't find on
the Bifferboard, others mentioned the /fastboot, even saying that the
file is removed on the next boot (which didn't happen). Others point
to tune2fs, which seems to leave room for other checks which I still
wanted to avoid ("last check is in the future, fixed, so going to
check again but don't go away because it will just take me a moment to
fail at it", or "last write was in the future, fixed, here we go
again", and so on).

So I finally got to the fstab, and I was glad to know there was this
general master disable switch for the forced disk checks. About one
hour later the disk check on the PC was done, so I tried it and it
worked. So it all ended up being less painful than I expected.

I should have tried the word "automatic"!!! Raios e coriscos!!!
(that's portuguese for "shucks!!") I'll remember that the next time I
have some other stupid problem! :-)

Thanks!

Andrew Scheller

unread,
Mar 19, 2012, 9:23:39 AM3/19/12
to biffe...@googlegroups.com
[snip]

If you're not familiar with Linux, starting out with the bifferboard
is going to be a very difficult learning environment, as you seem to
be discovering. It works quite differently from a normal desktop PC
running Ubuntu ;-D

Now that you're disks aren't getting checked at all, don't come crying
if you get unrecoverable filesystem-corruption ;)
I think if you're able to get rid of the daily-reboot, that'd be a
much better solution to some of your problems... but I can't offer any
google-search-terms suggestions for that one!

Lurch

Rilhas

unread,
Mar 19, 2012, 1:33:21 PM3/19/12
to Bifferboard
>
> If you're not familiar with Linux, starting out with the bifferboard
> is going to be a very difficult learning environment

The best horse for me is one that has a horn on its head and can fly.
And the best Christmas is one where Santa Claus exists and brings me
presents no matter how heavy or expensive, as long as I've been a good
boy (which my mother always said I am). And the ideal OS, for me, is
one for which I don't have to learn anything and still get it to work
flawlessly. People tell me that the Mac and iPhone are good examples
of such a fantastic concept... I still need more convincing to jump on
board, but I try to keep an open mind. :-) In the meantime I will try
to learn as little as possible about Linux specifically for the
Bifferboard while still getting it to work!(?) :-)

My final purpose for the Bifferboard is to use it as a platform for my
domotics system, which I intend to sell in the near future. All my
beta-testers love it, but they don't want to spend a lot of money on a
PC (or "equivalent") that will surely have reliability issues. The
Bifferboard addresses precisely those issues, but with each of these
problems I postpone it for a little longer fearing I might be wrong,
there would be nothing worse for my system's reputation than not
booting because a disk check failed. How would my automated
intelligent refrigerator know at what temperature it should keep my
food if the central processor went out because of a disk check
failure??? :-)

So I'm getting kind of frustrated as it seems I will be unable to make
peace with the Bifferboard. The reason why I chose the Bifferboard was
mainly because I thought that such a low demanding Linux platform (150
MHz, 32MB RAM) would not have heat problems, but as it turned out I
was wrong (as mentioned in one of my recent posts). However, I'm still
willing to give it a chance, but being unable to perform disk checks
is very close to being a show stopper. Not so much because of the
issue of being unable to do a specific disk check, but the realization
that disk checks that work today may begin to fatally fail later as
the number of files on the disk increases. I will have to search and
find a way to compute what these limits are exactly to see if they can
compromise or limit my system if it runs on a Bifferboard. Caching
filesystem checks to disk instead of using RAM is also not a good
option, a domotics system cannot be down for minutes, let alone hours.

> Now that you're disks aren't getting checked at all, don't come crying
> if you get unrecoverable filesystem-corruption ;)

What??? How could you say that without even a spoiler alert??!!! ...
now that I know how this story will end I'm not sure I want to stick
around to see it through... :-)

But fear not. Remember: I'm a survivor of a hack or disk corruption
(remember the post about my server being hacked a while back?), and,
even though I'm not sure which, that never stopped me from going over
to Fabio's and ring on his door bell at 4am because my Bifferboard has
a boo-boo. So I'll only need to come crying if Fabio decides it is
time to move and changes his phone number.

nicolas le falher

unread,
Mar 19, 2012, 1:39:26 PM3/19/12
to biffe...@googlegroups.com
>My final purpose for the Bifferboard is to use it as a platform for my domotics system, which I intend to sell in the near future
Why a domotic system need a 500gb hard disk ?

Nicolas

Andrew Scheller

unread,
Mar 19, 2012, 2:23:29 PM3/19/12
to biffe...@googlegroups.com
> In the meantime I will try
> to learn as little as possible about Linux specifically for the

Sorry to sound harsh, but I'm afraid that kind of attitude probably
won't get you many friends in the Linux community.

> My final purpose for the Bifferboard is to use it as a platform for my
> domotics system, which I intend to sell in the near future. All my

For sush an embedded system, you'd be much better off running
something like OpenWRT from the on-board flash, rather than running
(Debian Linux?) from a USB HardDrive.

> intelligent refrigerator know at what temperature it should keep my
> food if the central processor went out because of a disk check
> failure??? :-)

Nicolas already asked exactly the same question I was going to ask here ;-)

> mainly because I thought that such a low demanding Linux platform (150
> MHz, 32MB RAM) would not have heat problems, but as it turned out I
> was wrong (as mentioned in one of my recent posts). However, I'm still

As you already suggested in your other thread, I suspect this could be
a problem of heat being transferred from the USB HDD. And with all
that other equipment you have running in the same enclosed cupboard
too... things are bound to get a bit 'warm'

> issue of being unable to do a specific disk check, but the realization
> that disk checks that work today may begin to fatally fail later as
> the number of files on the disk increases. I will have to search and

As already explained, if you set things up in the traditional method
of a small root partition, and a separate (large) /home partition, you
probably wouldn't be seeing these kinds of out-of-memory problems.

> find a way to compute what these limits are exactly to see if they can
> compromise or limit my system if it runs on a Bifferboard. Caching
> filesystem checks to disk instead of using RAM is also not a good
> option, a domotics system cannot be down for minutes, let alone hours.

If the disk-check takes hours on your 'normal' PC, they'll obviously
take even longer on the low-power CPU in Bifferboard.
And if you need permanent long-term (and instant-reboot) reliability,
a microcontroller-based solution may be better than a Linux-based
solution.

> Now that you're disks aren't getting checked at all, don't come crying
>> if you get unrecoverable filesystem-corruption ;)
> What??? How could you say that without even a spoiler alert??!!! ...
> now that I know how this story will end I'm not sure I want to stick
> around to see it through... :-)

I meant that repeatedly mounting the HD and rebooting the Bifferboard
is more likely to give you disk errors, and if you're not running any
fscks you may not get to see any errors (however unlikely they are)
until it's too late...?

Good luck with the rest of your BB-projects :)

Lurch

Rilhas

unread,
Mar 19, 2012, 2:27:05 PM3/19/12
to Bifferboard
> Why a domotic system need a 500gb hard disk ?

Ah, that is very interesting question. I'm conviced the answer is
probably the same as that for the question "why does a millipede need
all those legs?". I'm not sure, but it may have something to do with
never having to worry about stumbling. But I digress.

At first I wanted 500GB to make a backup of the internet. But that was
taking too long so I had to find another use for my 500GB hard disk.
So I decided to store all my music, photos, and videos in it. And
version them in the SVN system just to be sure any eventual changes to
rhythm and tempo are reversible :-). I still don't know what to do
with the remaining 400GB though.

The domotics system is able to record images from the survaillance
cameras (although video intrusion detection has to be performed in a
much more powerful computer, and so, for now, it is outside the scope
of what I want to get running on the Bifferboard, maybe this will
endup having to go on an FPGA). Cameras can easily generate 10GB per
day per camera of "significantly useful data" when it is windy,
because recordings have to start before the detector decides if it
should issue an alarm or not. Right now I'm deleting the recordings
(because of other problems I still have to solve), but I should keep
them to constantly retrain the detector. So at any given time there
should be arround 100GB of recordings present in the system, old ones
constantly being deleted as new ones are constantly created, while
present ones are used for learning and training. I would like the
Bifferboard's harddisk to provide for this storage, if possible.

If I restrict the disk capacity discussion to the things that can be
run on the Bifferboard alone then video is not that important (I will
never be able to stream it through the 500Kbps radio link), although I
would still like it if a second hard disk was not required just for
that. So the best second use for the storage is the ability to store
music (to play through the domotics system), so it is interesting to
not have a limited storage capacity.

It is easy to note that music should not be a primary concern to the
domotics system, because you can play music several other ways. That
is true, but I find it very annoying to have to switch to another
system to do a different thing. Inflexibility is also a problem, my
domotics system is all about being able to do smart things (like
automatically turning down the music when you get a phone call, for
example, or when your wife is calling you from another room even if
you don't hear her calling), and that is hard to do with current
alternatives. That aplies even strongly to having multiple hard disks
and gadgets for multiple purposes. So, if my domotics requires a hard
disk to run, then it should be able to use a big disk to do NAS and
other things the user wants to do, and not force the user to have
other separate disks and devices "otherwise the domotics system might
break".

But for now the disk checks fail even with the disk filled only to
about 80GB or so. I know that with 8GB containing only the OS, the
webserver's files, and the e-mail files the checks succeeded (oh, here
are other usages for the 500GB: user's webservers and mailboxes), but
I don't know at which point the checks started failing because I
switched to 500GB and loaded the 80GB in a relatively small period,
without performing any disk checks. Only 25 days later did I realize
it as not booting anymore.

... and, of course, I forgot to mention: 500GB might not be enough for
all the logs I need in order to squash all the bugs I still have.
Users may find themselves needing them too because the system is
supposed to be highly programmable... :-)

Andrew Scheller

unread,
Mar 19, 2012, 2:40:24 PM3/19/12
to biffe...@googlegroups.com
>> Why a domotic system need a 500gb hard disk ?
>
> Ah, that is very interesting question. I'm conviced the answer is
> probably the same as that for the question "why does a millipede need
> all those legs?". I'm not sure, but it may have something to do with
> never having to worry about stumbling. But I digress.
[snip]

I'd suggest that with all you're trying to do, you may have much
better success (or at least less stumbling) if you use a system with a
faster CPU and much more memory than a Bifferboard? You do realise
that the bifferboard hardware was originally designed to be "just" a
cheap low-powered NAS, right? ;)
http://www.addonics.com/products/nasu2.php
http://blog.famzah.net/2009/11/25/bifferboard-performance-benchmarks/

At the very least, I'd recommend at least "developing" on a more
powerful system, and then once you have it all working, testing if it
can be "deployed" (i.e. squeezed) successfully onto the Bifferboard.

Lurch

Rilhas

unread,
Mar 19, 2012, 3:09:52 PM3/19/12
to Bifferboard
>
> Sorry to sound harsh, but I'm afraid that kind of attitude probably
> won't get you many friends in the Linux community.
>

Ohhh... come on... doesn't a guy have the right of not wanting to
learn about something?? I learned about cooking, shouldn't that be
enough? I'm good at it too!
I think it is like with my friends who don't know how mobile phones
work, most really don't care, but I still love them. You can try to
teach them, and they can fake an interest, or you
can even really get them to be interested for like a second or two,
but in the end there is no point to it.

As for the rest I would much rather continue to worship you and
apreciate all the magic and wonder of the things you know how to do
without having to become a wizard myself.

I think my soul is not yet completely lost, it's still good, it's
still good!! Look, I'm still trying to get my system to run in Linux,
right?

>
> For sush an embedded system, you'd be much better off running
> something like OpenWRT from the on-board flash, rather than running
> (Debian Linux?) from a USB HardDrive.


There it is! Did you see it?? Magic! :-)

... but I'll Google that OpenWRT thingy to try and understand.

>
> As you already suggested in your other thread, I suspect this could be
> a problem of heat being transferred from the USB HDD. And with all
> that other equipment you have running in the same enclosed cupboard
> too... things are bound to get a bit 'warm'

Yes, true. But keeping it closed in the Winter (<28ºC) is still colder
than keeping it open in the Summer (>30ºC), so keeping it closed
serves as a good indication of what the Summer will be like. I'm
hopping that keeping it naked is a good "solution", let's see how it
goes. If I can't keep it closed then I will be in trouble, because
zero-aesthetic impact in people's homes is requirement number one for
the domotics system, zero-noise being the second (so there go fans).


>
> As already explained, if you set things up in the traditional method
> of a small root partition, and a separate (large) /home partition, you
> probably wouldn't be seeing these kinds of out-of-memory problems.


I guess so, the permanent fix I mentioned (to do in 6 months! :-)) is
like that. But it is only a solution to the RAM problem because it
uses disk cache for checks, so I already know that there will still be
some impact on the system, especially the one where the impact is
strongly related to the number of files in the system, which can
surprise the users with a sudden (er even exponential) performance
degradation.

But I do agree with you, I don't think there is anyway around it at
this point.

> And if you need permanent long-term (and instant-reboot) reliability,
> a microcontroller-based solution may be better than a Linux-based
> solution.

The reboot can take a minute, or two, not 10. Most of the hardware
will enter a fallback mode if not contacted by the central processor
and they don't hear any radio chatter for X seconds. Most critical
functions, like lights and light switches, default to this mode after
30 seconds, but these are the easy ones. Closing or opening blinds are
trickier, if I'm in a hurry to leave the home I don't want to wait 2
minutes for the system to reboot until I can finally close the blinds.
For alarms I don't have a solution for server downtime yet, right now
the only solution I have is running a copy of the system on my mobile
phone through the internet (as a backup solution), but it don't know
how much of a security risk this is (but I think I can guess! :-)).

I don't think I can have a good platform for domotics if I run it on a
microcontroller. It is true that all the hardware of my system is
based on microcontrollers, but those will be small, hidden away, and
will just do what they are told to by the central system. They are not
truly intelligent. It is this central processor I want the users to
program and test on PC's, even debug on PC's using real hardware
already installed in their lamps and power outlets and irrigation
systems, and once it is ready then deploy to the Bifferboard and
forget. Complex things like deciding if I should water the lawn or not
by checking the internet for weather predictions to see if it is going
to rain sometime in the next 3 days are just too hard to do with
microcontrollers, or determining if tripping a PIR sensor should
generate a phone call to you or just turn on a light are also much
more complex than they seem (if you want them to be really
intelligent!). In my view when it comes to the intelligent decision-
making process the ratio between effort and payback is much better
with a "computer" than with a "microcontroller".


>
> Good luck with the rest of your BB-projects :)
>

I think I may need it! :-)

Rilhas

unread,
Mar 19, 2012, 3:20:54 PM3/19/12
to Bifferboard
>
> I'd suggest that with all you're trying to do, you may have much
> better success (or at least less stumbling) if you use a system with a
> faster CPU and much more memory than a Bifferboard? You do realise
> that the bifferboard hardware was originally designed to be "just" a
> cheap low-powered NAS, right? ;)http://www.addonics.com/products/nasu2.phphttp://blog.famzah.net/2009/11/25/bifferboard-performance-benchmarks/

Actually I didn't know that! I always thought the Bifferboard was born
to be the central processor to my domotics system, but your
explanation makes much more sense! :-)

>
> At the very least, I'd recommend at least "developing" on a more
> powerful system, and then once you have it all working, testing if it
> can be "deployed" (i.e. squeezed) successfully onto the Bifferboard.

Yes, I agree, that is what I do now. I develop under Windows, using
Visual Studio, and I even debug in Windows. And I have a Windows
laptop acting as domotics server, but that is all because the
Bifferboard still surprises me every now and then. But I've tried
running the domotics on a Bifferboard, and it works well, the software
itself is not very demanding, right now is the backup I have for when
the Windows laptop finally dies. Currently, my problem with the
Bifferboard is not with the domotics software, but with the OS itself.
And I realize that is mostly "my problem", but that has never stopped
me from complaining before.

So right now all I run on the Bifferboard has just been a test. I
still develop everything in Windows, and when done I then open up my
Ubuntu VM and build for the Bifferboard. The webserver, the e-mails,
and all the rest I have running on the Bifferboard have just been
tests for me to determine how often I would have been left locked out
of my house because the Bifferboard didn't boot, had I been running
the domotics in it. :-)

Andrew Scheller

unread,
Mar 20, 2012, 7:03:02 AM3/20/12
to biffe...@googlegroups.com
> Ohhh... come on... doesn't a guy have the right of not wanting to
> learn about something?? I learned about cooking, shouldn't that be
> enough? I'm good at it too!

I can't tell if you're joking or if you're being serious... :-/

> I think my soul is not yet completely lost, it's still good, it's
> still good!! Look, I'm still trying to get my system to run in Linux,
> right?

Yeah, at least you haven't asked how to run Windows on the Bifferboard yet ;)

>> For sush an embedded system, you'd be much better off running
>> something like OpenWRT from the on-board flash, rather than running
>> (Debian Linux?) from a USB HardDrive.
>
> There it is! Did you see it?? Magic! :-)
> ... but I'll Google that OpenWRT thingy to try and understand.

Or you could just read the Bifferboard wiki...
https://sites.google.com/site/bifferboard/Home/openwrt-git
https://sites.google.com/site/bifferboard/Home/factory-firmware/newbie-instructions

The more you learn, the less things seem like magic :)
(the factory-firmware is simply an older version of Biff's openwrt-git
fork of http://openwrt.org )

> intelligent!). In my view when it comes to the intelligent decision-
> making process the ratio between effort and payback is much better
> with a "computer" than with a "microcontroller".

I think if I was in your situation I'd try a hybrid layered approach,
combining microcontrollers for critical front-end systems (
http://jeelabs.com/products/jeenode seems designed for exactly that
kind of thing) like temperature controls and door-alarms etc., a
bifferboard running from embedded-flash for higher-level logic /
internet access, and a dedicated NAS for high-performance "large"
file-storage.
So if the NAS goes down for a disk-check, the bifferboard will still
keep running, and if the bifferboard goes down for a reboot, your
fridge won't melt and you'll still be able to open your front door.
And I'd even suggest running non-critical things like a SVN server and
email on second bifferboard, rather than on the same bifferboard that
controls your door locks!

But hey, only you can know how your domotics will work best and
exactly what it should do...

Lurch

rolf

unread,
Mar 20, 2012, 7:53:33 AM3/20/12
to Bifferboard
On 20 Mrz., 13:03, Andrew Scheller <ya...@loowis.durge.org> wrote:

> I think if I was in your situation I'd try a hybrid layered approach,
> combining microcontrollers for critical front-end systems (http://jeelabs.com/products/jeenodeseems designed for exactly that
> kind of thing) like temperature controls and door-alarms etc., a

Hi,

that's just what I do. But for homecontrol applications bifferboard is
just fine, because via GPIO a lot of sensors can be driven and with
linux you don't need to do much software development. Exeptions are
just time critical applications, which I do not have, distant
measuring points and a lot of sensors. Just for the latter two I use
jeenodes, wireless, with a jeelink-USB-stick plugged into the
bifferboard-server.

If one needs really a powerful but small and lowcost server, I
recommend this one:
http://www.plugcomputer.org/development-kits/plug-computer-basic.html
It looks nice, but I do not need it, never pushed the bifferboard to
its limits yet.

Greetings,

Rolf

Rilhas

unread,
Mar 20, 2012, 8:19:21 AM3/20/12
to Bifferboard
> I think if I was in your situation I'd try a hybrid layered approach,
> combining microcontrollers for critical front-end systems (http://jeelabs.com/products/jeenodeseems designed for exactly that
> kind of thing) like temperature controls and door-alarms etc.

I was joking about the refrigerator, my system has all the control
electronics (with microcontrollers) distributed around the house and
the Bifferboard only has a radio connected to it to controll
everything. And, of course, each controller has its fallback mode to
operate when the central processor is down. By the way, that is what
is taking time, what I intend to sell is basically the control
hardware and a low level API, for the rest (framework and central
processing software) I will leave it up to the individual users and
the community.

> bifferboard running from embedded-flash for higher-level logic /
> internet access, and a dedicated NAS for high-performance "large"
> file-storage.

That is what I was thinking when I first bought the Bifferboard, but I
then realised that it is not practical to have that setup because
updates to the software are too frequent for that. For example, in
this first year I updated the software more than 100 times, so while
it is practical to replace the executable and restart the application
(10 seconds) it would not be practical take the Bifferboard out of the
closet, flash it, and then put it back (not so much the downtime but
the nuissance).

I still want to use it in the embedded flash but only when it is
stable enough for me. Other users will have it stable for them at
different times, so I think the distribution must account for an
initial hard disk or USB flash solution, so as to be practical.

>
> But hey, only you can know how your domotics will work best and
> exactly what it should do...
>

I think that sums up what the big problem in domotics is all about,
and it explains why it has been so hard for that market to expand.
Although I'm pretty much convinced I found a good solution for me I
already verified that other users will want other things, thus what I
feel is the need to keep it flexible and practical.

So, I think you just hit the spot! :-)

Lukasz Sokol

unread,
Mar 20, 2012, 8:36:17 AM3/20/12
to biffe...@googlegroups.com
Hello Rilhas,

basically +1 on what Andrew said, (including the joking)...

plus my �0.01...


>>
>> As you already suggested in your other thread, I suspect this could be
>> a problem of heat being transferred from the USB HDD. And with all
>> that other equipment you have running in the same enclosed cupboard
>> too... things are bound to get a bit 'warm'
>

> Yes, true. But keeping it closed in the Winter (<28�C) is still colder
> than keeping it open in the Summer (>30�C), so keeping it closed


> serves as a good indication of what the Summer will be like. I'm
> hopping that keeping it naked is a good "solution", let's see how it
> goes. If I can't keep it closed then I will be in trouble, because
> zero-aesthetic impact in people's homes is requirement number one for
> the domotics system, zero-noise being the second (so there go fans).
>

naked as in 'no plastic cover' may have its reasons too, but if I were
doing a thing like that, I would definitively /not/ squeeze the file server
into the same system as domotics;
You may even keep more than one BB in one cupboard, but ventilation is a must
(side note - do you keep the systems' PSU in the same cupboard too ?)

>
>>
>> As already explained, if you set things up in the traditional method
>> of a small root partition, and a separate (large) /home partition, you
>> probably wouldn't be seeing these kinds of out-of-memory problems.
>
>
> I guess so, the permanent fix I mentioned (to do in 6 months! :-)) is
> like that. But it is only a solution to the RAM problem because it
> uses disk cache for checks, so I already know that there will still be
> some impact on the system, especially the one where the impact is
> strongly related to the number of files in the system, which can
> surprise the users with a sudden (er even exponential) performance
> degradation.
>
> But I do agree with you, I don't think there is anyway around it at
> this point.
>
>> And if you need permanent long-term (and instant-reboot) reliability,
>> a microcontroller-based solution may be better than a Linux-based
>> solution.
>
> The reboot can take a minute, or two, not 10. Most of the hardware
> will enter a fallback mode if not contacted by the central processor
> and they don't hear any radio chatter for X seconds. Most critical
> functions, like lights and light switches, default to this mode after
> 30 seconds, but these are the easy ones. Closing or opening blinds are
> trickier, if I'm in a hurry to leave the home I don't want to wait 2
> minutes for the system to reboot until I can finally close the blinds.

First, if you get rid of the NAS function on the same BB, the reboots
will shorten, second, they /may/ not be needed at all (only if power fails?)

> For alarms I don't have a solution for server downtime yet, right now
> the only solution I have is running a copy of the system on my mobile
> phone through the internet (as a backup solution), but it don't know
> how much of a security risk this is (but I think I can guess! :-)).

[...]


> Complex things like deciding if I should water the lawn or not
> by checking the internet for weather predictions to see if it is going
> to rain sometime in the next 3 days are just too hard to do with
> microcontrollers, or determining if tripping a PIR sensor should
> generate a phone call to you or just turn on a light are also much
> more complex than they seem (if you want them to be really
> intelligent!).

That's almost probably too demanding for /single/ BB as well.

Having /more/ split system will ultimately work better as e.g. one BB
would drive lamps/electrics, other will drive blinds/gates etc, you
could even do a fancy 'fallback' redundancy ( so that any BB in array
could take over tasks of any other if required) and the lot will
be all more fool- and disaster-proof.

(Oh, check out OpenMosix, a system of yesterdecade, before dawn of the multi-core
processors somebody devised a system that could turn an array of computers
into a cluster without need for the programs to be compiled specially for
cluster awareness... the creator has abandoned it (sadly) but others moved on
and created LinuxPMI - wonder if THIS could run on the BB ? your 'head' application
could work distributively on ALL bb's in array, and the individual sub-system bb's
only be 'in charge' of one task (but, could serve as fallback protection in case
one fails/decides to reboot/whatever)

> In my view when it comes to the intelligent decision-
> making process the ratio between effort and payback is much better
> with a "computer" than with a "microcontroller".
>
>
>>
>> Good luck with the rest of your BB-projects :)
>>
>
> I think I may need it! :-)
>

<Semi-joke> question : are you in such a need to have x86 compatible
processor in your tiny 'main CPU' of the system that you can't choose
other product family, e.g. some tiny board containing an ARM processor
(point in case: Raspberry PI) that has mighty more RAM and faster CPU?
There are cross-compilers available for most software development systems
(e.g. Lazarus/FPC can compile for ARM) As a bonus with Raspberry PI you
get a graphics chip so you could have a touchscreen application even...
(and ARM cpu's are notoriously cold even at speed)
</semi-joke>
(thou I don't know about OpenMosix/LinuxPMI status on that, and still for domotics
redundancy is a good idea (tm) if not a must (r))

Not that I'd recommend this on a BifferBoard list, oh no :)
L.

Lukasz Sokol

unread,
Mar 20, 2012, 8:59:45 AM3/20/12
to biffe...@googlegroups.com
On 20/03/2012 12:19, Rilhas wrote:
>> I think if I was in your situation I'd try a hybrid layered approach,
>> combining microcontrollers for critical front-end systems (http://jeelabs.com/products/jeenodeseems designed for exactly that
>> kind of thing) like temperature controls and door-alarms etc.
>
> I was joking about the refrigerator, my system has all the control
> electronics (with microcontrollers) distributed around the house and
> the Bifferboard only has a radio connected to it to controll
> everything. And, of course, each controller has its fallback mode to
> operate when the central processor is down. By the way, that is what
> is taking time, what I intend to sell is basically the control
> hardware and a low level API, for the rest (framework and central
> processing software) I will leave it up to the individual users and
> the community.
>
Ok so I might have under-rated/estimated you in my prev. email... sorry :)
no offense meant there.
But still IMO having a clustered bb solution for the 'head controller'
could save you a lot of bother.

>> bifferboard running from embedded-flash for higher-level logic /
>> internet access, and a dedicated NAS for high-performance "large"
>> file-storage.
>
> That is what I was thinking when I first bought the Bifferboard, but I
> then realised that it is not practical to have that setup because
> updates to the software are too frequent for that. For example, in
> this first year I updated the software more than 100 times, so while
> it is practical to replace the executable and restart the application
> (10 seconds) it would not be practical take the Bifferboard out of the
> closet, flash it, and then put it back (not so much the downtime but
> the nuissance).

Running the minimally booting OS from on-board flash, the core OS parts
and the control application from USB flash, (and leave the spinning
HDD alone to die, damn :) ) could quite surprise you I think.
(the app could check external server for new version and update itself alone)

>
> I still want to use it in the embedded flash but only when it is
> stable enough for me. Other users will have it stable for them at
> different times, so I think the distribution must account for an
> initial hard disk or USB flash solution, so as to be practical.
>
>>
>> But hey, only you can know how your domotics will work best and
>> exactly what it should do...
>>
>
> I think that sums up what the big problem in domotics is all about,
> and it explains why it has been so hard for that market to expand.
> Although I'm pretty much convinced I found a good solution for me I
> already verified that other users will want other things, thus what I
> feel is the need to keep it flexible and practical.
>
> So, I think you just hit the spot! :-)
>

It's the same with industrial type applications : these are things that
affect production - people want production to go on even if an output transistor
on the controller output module has blown, What affects production, affects
money... hence they 'invented' redundancy and if something has to go through
single computer, it has to have $million$ guarantee from its producer that it's
rock-solid... thence they tend to use very very old proven software for that :J
(I have seen a industral flatbed 2 axis wood router still operating on Win3.1 in 2005...)
Domotics isn't fundamentally different from that I think.

L.

Rilhas

unread,
Mar 20, 2012, 9:33:52 AM3/20/12
to Bifferboard
> Hi,
>
> that's just what I do. But for homecontrol applications bifferboard is
> just fine, because via GPIO a lot of sensors can be driven and with
> linux you don't need to do much software development. Exeptions are
> just time critical applications, which I do not have, distant
> measuring points and a lot of sensors. Just for the latter two I use
> jeenodes, wireless, with a jeelink-USB-stick plugged into the
> bifferboard-server.

I do have time critical situations. The most critical are (believe it
or not) when the users turn on a light switch and want the lamp to
turn on right away (less than 0.2 seconds otherwise they complain) and
when they press a button on the remote control and want the TV and
stereo and set top box must all ajust within 1 second. Following these
two "emergencies" is turning on lights immediately when passing
through a PIR sensor, but this I found can take 0.5 seconds and it is
still acceptable. Other things, like blinds control, HVAC control,
door bell control, door lock control, etc. are all in the range of 5
seconds of response time.

To complicate matters lamp switches are modified to spring back, so
they behave just like mouse buttons. From then on you have weird new
options like click the light switch, double click it, long click it...
clicking might turn it on and off, double clicking might turn all the
lamps on or off, and long clock may fade in or turn off in 2 minutes,
for example. The speed at which people get feedback on these
operations has to be really really fast otherwise they complain.

And all this logic is not tied to the microcontrollers, otherwise
there goes flexibility out the window. So, when a user presses a light
switch a message is sent to the Bifferboard, which immediatelly
decides if it should turn on the associated lamp and send a message to
it to turn it on (since the protocol is confirmed this results in the
exhange of 4 messages in 0.2 seconds, 20 messages per second).

That is where the fallback comes in. If the lamp switch doesn't get an
answer from the Bifferboard after the user presses a switch in X
seconds than it sends a message to the previously associated lamps
(stored in its EEPROM) and the system is then inflexible but is still
able to turn the lights on or off.

Anyway, all these limits are well within the Bifferboard's
capabilities, I tested the round-trip extensively and the Bifferboard
can do 40 messages per second and takes less than 1ms to wake up from
its message loop and perform the necessary decisions. Other long
blocking operations (like crawling the weather website) are run on
separate threads.

>
> If one needs really a powerful but small and lowcost server, I
> recommend this one:http://www.plugcomputer.org/development-kits/plug-computer-basic.html
> It looks nice, but I do not need it, never pushed the bifferboard to
> its limits yet.
>

Initially I was planning on using a SheevaPlug, whose specs are very
similar to the ones of the computer you mentioned, but it had a lot of
reliability issues, especially with the power supply and temperature.
So I kept on searching, and that is when I found the Bifferboard
(well, I didn't actually find it, a colleague told me about it on a
lunch break when I was telling him about the SheevaPlug's problems and
what I wanted to do with it) .

I would like to point out that there is no need for a powerful control
computer in my domotics system, so having a more powerful computer
should not be necessary. The Bifferboard is already overkill, 40
messages per second and 1ms response time is way more than enough. The
32MB of RAM is also enough, the peak is 5MB of usage, and that is
while crawling websites.

I have been pushing the Bifferboard's limits, and I've tried running
my webserver, POSTFIX, DOVECOT, the TV and movies crawlers, IP
updater, HTTP router, SVN server, and log compression, all together
with the domotics server software (which has its own webserver and a
server for Android, Windows Mobile, and J2ME phones), and none of the
applications ran into problems. Well... maybe SVN, but then I
increased the swap file size to 200MB and the problem went away. Of
course, my webserver is not visited by 1000 users per second, and
there is only so much I can SVN-commit at any given time, but it all
works just fine!

So my posts are not about the Bifferboard's performance, I find that
to be more than adequate. It is all about getting to configure Linux
so as to not cause any problems. Although I don't really know Linux,
I've been told numerous times that one of its best features is
flexibility - the ability to get it just like I want it.

You have probably noticed I'm really fighting to work around the disk
check problems in a way that doesn't affect performance, so that I may
keep the Bifferboard. In my view it would be a shame if I had to
change to another computer just because it can't check the disk with
only 32MB of RAM! Even if the disks were just fine! And, of course, I
would have to start the 1-year tests with another computer all over
again!

Anyway, I don't know if 512MB of RAM would improve things
significantly. When I first had this problem I read a post from some
guy who had run tests and got a figure like 5MB of RAM (peak usage)
per 1GB of harddisk. If the guy is right then 512MB of RAM would be
enough to check only 100GB of hard disk, which (for me) is still not a
definitive solution, and disk checking (whether in RAM or cached in
disk) would still be a problem, even with the 2-partition scheme,
because performance would be significantly and/or exponentially
affected if RAM is not enough, no matter which computer.

I don't mean to offend any of Linux's hardcore fans, but I've been
running Windows machines for 20 years and I have had 0 fatal disk
corruptions (yes, read as zero). That is hundreds of machines going by
my hands, and that includes FAT16 Windows 95. Sure, files got lost (in
FAT16 they disappeared or got mangled at an impressive rate) but the
system could always start, even if crippled. I don't know if things
were somehow being fixed in the background or what, ir if critical
system files were in any way backed up, but it booted. Windows "safe
mode" was always very good at getting it to boot no matter what was
wrong, and Windows 95 didn't need that many files to boot.

Back in the day I also fixed failing harddisk for friends, and I did
get many failing disks, but I'm excluding these from the statistics,
obviously. I think I might have been lucky getting a number as low as
zero (zero is a really low low number!!), but all around me the number
of cases of people having corrupted non-failing unbootable disks,
although not zero, was low. Looking back at a drawer I have full of
irrecoverably damaged hard disks I kept from those days the only
conclusion I can take is that disks were failing much more often than
file systems got corrupted. :-)

Then, about 15 years ago, I needed to really test filesystem
reliability (for work). So I performed all kinds of torture on 20 NTFS
disks (I didn't know about waterboarding back then) and none refused
to boot afterwards, always without any checking or fixing.

I don't really attribute this high reliability to Windows itself.
Maybe I have to admit that NTFS is good, so ok, I admit it, I have
made enough tests to prove it, so I have no way around that, but I
don't really attribute any merit to Windows itself. So I want to get
the same reliability out of Linux, without the need to constantly
check.

I know that when Windows PC's had 32MB of RAM disks were only 1GB or
so, so I shouldn't automatically expect the 32MB Bifferboard to be
able to handle a 500GB harddisk as easily. Granted, I know this. And I
do know the Bifferboard can check an almost full 8GB flash disk, so I
don't really blame the Bifferboard. But still, I need a practical
solution.

So what I want is to be able to run the server computer on a
filesystem that doesn't get corrupted no matter how unexpectedly
applications die or how many times the power goes out while data was
still being written, regardless of being run on the Bifferboard or any
other computer. Changing computer does not address this issue, it just
works around it (unless I can only get a reliable filesystem if I use
a computer other than the Bifferboard, is that the case?). I know that
this is possible, I verified it personally with NTFS, so I believe it
is possible with Linux too. If not Ext3, then Ext4? If so then that is
what I want on my Bifferboard, no need to change to another computer.
That would solve all remaning problems I have with the Bifferboard
(well, except for the overheating, but I will have to wait for the
Summer to come and go to be able to confirm). At that point I would be
able to go ahead and retire my Windows laptop, and possibly start
marketing my domotics system with the Bifferboard as the server-side
companion! :-)

So I'll just keep on trying, I'll stop if I hit a brick wall! :-)

Rilhas

unread,
Mar 20, 2012, 9:42:42 AM3/20/12
to Bifferboard
>
> Ok so I might have under-rated/estimated you in my prev. email... sorry :)
> no offense meant there.

No worries! :-)

> But still IMO having a clustered bb solution for the 'head controller'
> could save you a lot of bother.

Possibly... I think I should look into that possibility, thanks!

> Running the minimally booting OS from on-board flash, the core OS parts
> and the control application from USB flash, (and leave the spinning
> HDD alone to die, damn :) ) could quite surprise you I think.
> (the app could check external server for new version and update itself alone)

I see your point, running it from FLASH (even if getting an updated
version from the hard disk) so that its performance were not affected
by the sluggish HDD response while it was being checked. I think that
is a good idea! How would I keep the swap file (which has to be big
and be kept on the hard disk) from affecting performance? Can I
somehow tell Linux that my domotics executable is not to be swapped
out?

> It's the same with industrial type applications : these are things that
> affect production - people want production to go on even if an output transistor
> on the controller output module has blown, What affects production, affects
> money... hence they 'invented' redundancy and if something has to go through
> single computer, it has to have $million$ guarantee from its producer that it's
> rock-solid... thence they tend to use very very old proven software for that :J
> (I have seen a industral flatbed 2 axis wood router still operating on Win3.1 in 2005...)
> Domotics isn't fundamentally different from that I think.

... I was trying to avoid saying such things about the Bifferboard,
but I admit, its low speed CPU and simpler hardware attracted me much
more than 1GHz+ alternative solutions that were already available
exactly because of that. It was either that or a Z80... :-)

Lukasz Sokol

unread,
Mar 20, 2012, 11:26:34 AM3/20/12
to biffe...@googlegroups.com
On 20/03/2012 13:42, Rilhas wrote:
>>
>> Ok so I might have under-rated/estimated you in my prev. email... sorry :)
>> no offense meant there.
>
> No worries! :-)
>
>> But still IMO having a clustered bb solution for the 'head controller'
>> could save you a lot of bother.
>
> Possibly... I think I should look into that possibility, thanks!
>
>> Running the minimally booting OS from on-board flash, the core OS parts
>> and the control application from USB flash, (and leave the spinning
>> HDD alone to die, damn :) ) could quite surprise you I think.
>> (the app could check external server for new version and update itself alone)
>
> I see your point, running it from FLASH (even if getting an updated
> version from the hard disk) so that its performance were not affected
> by the sluggish HDD response while it was being checked. I think that
> is a good idea! How would I keep the swap file (which has to be big
> and be kept on the hard disk) from affecting performance? Can I
> somehow tell Linux that my domotics executable is not to be swapped
> out?

Having 32MB of RAM, you can't really avoid programs to go out to swap
(unless the program is so trivial it doesn't need it).
The way around that would be - separate partition for swap, on the rotating
HDD. (as swapping action could severely deteriorate 'cheap' flash USB sticks...)
As far as I got around reading lkml, the Linux guys tend to think, that memory
unused is memory wasted, so (I think) default swapper behavior is to swap as
little as possible (and certainly don't swap fs buffers, for example).
And out of reading Linux scheduler stories I get the impression, that
it'll eventually learn that the smaller process (program) that talks to the
network 20 times per second should not be swapped out even when the bigger (apache?)
lags (of course the scheduler could be told that by manipulating priorities appropriately)
(from reading your posts I get the impression you /want/ to know something about Linux ;)

>> It's the same with industrial type applications : these are things that
>> affect production - people want production to go on even if an output transistor
>> on the controller output module has blown, What affects production, affects
>> money... hence they 'invented' redundancy and if something has to go through
>> single computer, it has to have $million$ guarantee from its producer that it's
>> rock-solid... thence they tend to use very very old proven software for that :J
>> (I have seen a industral flatbed 2 axis wood router still operating on Win3.1 in 2005...)
>> Domotics isn't fundamentally different from that I think.
>
> ... I was trying to avoid saying such things about the Bifferboard,
> but I admit, its low speed CPU and simpler hardware attracted me much
> more than 1GHz+ alternative solutions that were already available
> exactly because of that. It was either that or a Z80... :-)
>

This 'head control' program of yours - I wonder - is it split into different modules e.g.
receiving input messages, processing, setting outputs ? The different parts of it, what are they?
Native executables or scripts (yeah perl or python is a script, not a program, face it ;) ) ?
Reading input messages and saving to kind of database (even text file is a database of sorts)?
Is a web-server used to present system state (as a website?) something like SCADA system ?

Is your fallback in domotics switches/actuators able to find the 'head processor' when it
comes back online? Can the domotics inputs/outputs be configured to find /another/ head if one
seems to have failed? Could the head send a message to peripherals 'oh guys, here, my brother's
rebooting, now I'm in charge' ? Is this kind of switchover within the responsiveness limits?

Maybe you don't need to spend time and energy on cluster solution (more so since the OpenMosix
and its successor operated at process level, not at application thread level, so if there is some
extensive processing the 'head control' program(s) has to be made so to make use of SMP multiprocessor
environment).

Maybe it's enough to have 2 BB's per system, that way you will able to achieve 99.99+% head uptime.
Maybe it's enough to have just a clever master/shadow switchover/hand-over system.
The 'shadow' BB would just mimic what the 'master' does to maintain consistent state (i.e.
receive and process the same input messages to produce same output, but not be allowed to talk
to outputs if it sees the master's still here, but switch over to talking to them when the master
bails out, effectively becoming master (and the other one when reboot is complete, will become shadow))

There is whole science behind the 'reliability of systems' issue :)

Are you able to /abandon/ the dream of running NAS on it as well as control? :)

L.

Rilhas

unread,
Mar 20, 2012, 1:17:18 PM3/20/12
to Bifferboard
> > Ohhh... come on... doesn't a guy have the right of not wanting to
> > learn about something?? I learned about cooking, shouldn't that be
> > enough? I'm good at it too!
>
> I can't tell if you're joking or if you're being serious... :-/
>

Actually, I was joking. My cooking skills are not that great! :-)

As for the rest, I have felt, at times, that many in the Linux
community present an odd "equivalency requirement" where Linux wizards
only "like" other Linux wizards. I should point out that I have never
felt that in this forum, maybe that is the reason I keep comming back
with my multi-page ramblings. Maybe many members of the forum clench
their teeth when they see me write that I don't want to learn Linux
but endure your suffering in silence? I hope not!

This opinion of mine is mostly based on observing my work colleagues,
and having watched some of their fights about Linux being better than
Windows, or JAVA being better than C#, or red being better than green
(footbal club colors), or whatever.

There is something "weird" about the Linux vs. Windows issue (just so
that I don't call it the "Linux issue") that I don't see in other
agressive discussions, like JAVA and C#, or red vs. green. A
discussion between JAVA and C# fans can last for hours, or even days,
often with a good mood between adversaries. The discussions on Linux
vs. <any other> tended to last less than an hour, but the underlying
agressivness would last far longer into the following weeks, often
hurting subsequent social behaviour between parties. I don't know why.
I have a number of theories, but none explains all that I have seen.

I have a friend who is a surf wizard (judging by the sponsorship level
she gets), but she is very comfortable with me not wanting to learn
how to surf. Having tried to teach me 1 or 2 times, se was fine with
me saying "I just want to learn enough to surf one wave, just one". It
doesn't really matter if I endedup unable to surf even one wave,
because she was really fine with me not wanting to learn that, and we
still had a good time trying.

Another friend tried to teach me how to make pudding (the dessert). I
explicitly said "there is absolutely no way I will ever want to learn
how to do pudding, because I love it so much that I would endup
puddiing myself to death". He insisted, he told me it was really easy,
he even shouted the recipe out loud to try to overcome my "NA-NA-NA-NA-
NA-fingers-in-ears" noises, but he didn't succeed. He was not happy,
he really wanted to tell me how it is done, but he was over it in a
minute and we are still friends.

But with Linux it is not like that, wizards really seem to be offended
by those (like me) that don't really want to learn, the ones who just
want to learn enough to use it as a tool. That is what I did when I
bought a hammer, I chose its weight, grip, handle length, head shape,
and even color, and then moved on with my life hammering away. I would
be said to know that the "hammer design experts" would be offended by
me not wanting to know.

I have nothing against Linux per se, I would like to be able to treat
it just like I do Windows. After 20 years of using Windows I really
know very little about Windows, and that is why I like it, it doesn't
require me to learn anything. And when I did have to learn (like
trying to start a program with a keyboard combination) I Googled it
and happily forgot all about it in 5 minutes.

Granted, I should not address Linux the same way as I do Windows,
Linux does require that you know a minimum. I think I've learned that,
so I now consider myself a Linux user! :-) ... Fabio helped, he was
kind enough to name the script that starts my stuff
"autoexec.sh"... :-)

I think the Linux community (in general) would be better off if it
weren't so... elitist? Is that a good word? (am I being
offensive?? ... sure hope not, it is not my intention!). Imagine if
Spielberg only liked to show his movies to people who knew how to
direct movies? Or even worse: to people who were good directors or
wanted to become good directors? Wouldn't the rest of us (including
people that don't care and don't want to learn how movies are made)
endup losing? And would Spielberg ever really get anything good from
that? Wouldn't the movie industry be much less advanced if all the
major directors limited access to their work to movie direction
students and enthusiasts? I sure think so, I kid you not!

Anyway, I would like to point out again I have never seen that in this
forum (not in messages exchanged with me nor with messages that I've
seen going by), and I think that is great! And Fabio tolerates me,
even after finding out that I don't want to learn about Linux more
than I need to (just in case I'll plant tracking devices on his dogs,
so if he decides to move I'll know where he will be!). :-)

Andrew Scheller

unread,
Mar 20, 2012, 1:27:02 PM3/20/12
to biffe...@googlegroups.com
> it is practical to replace the executable and restart the application
> (10 seconds) it would not be practical take the Bifferboard out of the
> closet, flash it, and then put it back (not so much the downtime but
> the nuissance).

Huh? No need to take it out of the closet and reflash the whole OS -
just SCP a new binary to it and write it the the JFFS2 partition.
OpenWRT is designed for embedded devices, so runs without swap (i.e.
without a HDD), but this obviously means it can't run stuff that is
"too big".

> for example. The speed at which people get feedback on these
> operations has to be really really fast otherwise they complain.
>
> And all this logic is not tied to the microcontrollers, otherwise
> there goes flexibility out the window. So, when a user presses a light
> switch a message is sent to the Bifferboard, which immediatelly

What if you had different "modes" programmed into the lightswitch
microcontroller, and bifferboard just told the microcontroller which
button-press-actions to trigger which light-switching-mode. Would be
quicker than sending every single button press backwards and
forwards... and does the 40 messages per second mean you're not
allowed more than 40 lightswitches in your house ;-D

> I have been pushing the Bifferboard's limits, and I've tried running
> my webserver, POSTFIX, DOVECOT, the TV and movies crawlers, IP
> updater, HTTP router, SVN server, and log compression, all together
> with the domotics server software (which has its own webserver and a
> server for Android, Windows Mobile, and J2ME phones), and none of the
> applications ran into problems. Well... maybe SVN, but then I
> increased the swap file size to 200MB and the problem went away. Of
> course, my webserver is not visited by 1000 users per second, and
> there is only so much I can SVN-commit at any given time, but it all
> works just fine!

Just because you *can* run everything-and-the-kitchen-sink on a single
bifferboard, doesn't necessarily mean it's a *good* idea... expecially
if you're worried about response-times.

> I don't mean to offend any of Linux's hardcore fans, but I've been
> running Windows machines for 20 years and I have had 0 fatal disk
> corruptions (yes, read as zero). That is hundreds of machines going by
> my hands, and that includes FAT16 Windows 95. Sure, files got lost (in
> FAT16 they disappeared or got mangled at an impressive rate) but the

Some would classify "files getting lost" as "fatal corruption".

> a computer other than the Bifferboard, is that the case?). I know that
> this is possible, I verified it personally with NTFS, so I believe it
> is possible with Linux too. If not Ext3, then Ext4? If so then that is

Just like NTFS, Ext3 and Ext4 are journalling filesystems. But
obviously no system can ever claim to be 100% reliable. There's always
lightning storms, solar flares, earthquakes, etc. ;-)

> (well, except for the overheating, but I will have to wait for the
> Summer to come and go to be able to confirm). At that point I would be

I suspect that as they're mechanical devices, harddrives probably
dislike excess heat even more than the Bifferboard.

Lurch

Lukasz Sokol

unread,
Mar 20, 2012, 1:32:45 PM3/20/12
to biffe...@googlegroups.com
On 20/03/2012 17:17, Rilhas wrote:
[...]

> But with Linux it is not like that, wizards really seem to be offended
> by those (like me) that don't really want to learn, the ones who just
> want to learn enough to use it as a tool. That is what I did when I
> bought a hammer, I chose its weight, grip, handle length, head shape,
> and even color, and then moved on with my life hammering away. I would
> be said to know that the "hammer design experts" would be offended by
> me not wanting to know.
>
[...]
Out of context, maybe but:
one now very rich man that started in computers once had to withdraw his
production line because of (!) something he didn't understand in the product...


That man was A.M Sugar, the founder of Amstrad...
the product they did not understand was... HDD storage had some issues.

Fail?
L.

Rilhas

unread,
Mar 20, 2012, 1:37:54 PM3/20/12
to Bifferboard
> > I see your point, running it from FLASH (even if getting an updated
> > version from the hard disk) so that its performance were not affected
> > by the sluggish HDD response while it was being checked. I think that
> > is a good idea! How would I keep the swap file (which has to be big
> > and be kept on the hard disk) from affecting performance? Can I
> > somehow tell Linux that my domotics executable is not to be swapped
> > out?
>
> Having 32MB of RAM, you can't really avoid programs to go out to swap
> (unless the program is so trivial it doesn't need it).
> The way around that would be - separate partition for swap, on the rotating
> HDD. (as swapping action could severely deteriorate 'cheap' flash USB sticks...)
> As far as I got around reading lkml, the Linux guys tend to think, that memory
> unused is memory wasted, so (I think) default swapper behavior is to swap as
> little as possible (and certainly don't swap fs buffers, for example).
> And out of reading Linux scheduler stories I get the impression, that
> it'll eventually learn that the smaller process (program) that talks to the
> network 20 times per second should not be swapped out even when the bigger (apache?)
> lags (of course the scheduler could be told that by manipulating priorities appropriately)
> (from reading your posts I get the impression you /want/ to know something about Linux ;)

You got me, I do want to learn about Linux :-), but just the
minimum... the rest of Linux I want to learn about I want it to come
from genuine interest (if and when it happens to appear) instead of
being forced to it because something is going wrong and the system is
down.

>
> This 'head control' program of yours - I wonder - is it split into different modules e.g.
> receiving input messages, processing, setting outputs ? The different parts of it, what are they?

For now it is one big executable, but modules can be easily split into
shared objects. In that case they become really small (10K->20K),
because generic framework stuff is linked against the executable
(which exports all its features). The webserver I use is already split
into small SO's, so I was able to verify this works well.

> Is your fallback in domotics switches/actuators able to find the 'head processor' when it
> comes back online? Can the domotics inputs/outputs be configured to find /another/ head if one
> seems to have failed? Could the head send a message to peripherals 'oh guys, here, my brother's
> rebooting, now I'm in charge' ? Is this kind of switchover within the responsiveness limits?

Yes, redundancy would be very easy to acomplish, hardware installed in
the house's controllable items can easily be taken over by secondary
BB's. Since they can't share resources (double the hard disks, double
the radios, double the Bluetooth range extenders, double everything
else) it is likely that I prefer reliability first, but with a second
BB standing by to be manually replaced if the primary one fails. The
same reasoning aplies to the hard disk, having a pen disk as emergency
backup. The radio can have a cheaper backup (the primary radio is
rather expensive, but its special capabilities are usually not
essencial). The rest of the components don't need backups, if the
Bluetooth extender fails you use 3G, if 3G fails.

You are giving me a lot to think about, I've been focusing on the "YOU
SHALL NOT FAIL!!" approach but I will eventually have to deal with the
fact that, no matter what, some will eventually fail. Anyway, each
user will determine his/her redundancy and fault tolerance levels, for
me I have a backup that runs on my phone, so I'm covered.

>
> Maybe it's enough to have 2 BB's per system, that way you will able to achieve 99.99+% head uptime.
> Maybe it's enough to have just a clever master/shadow switchover/hand-over system.
> The 'shadow' BB would just mimic what the 'master' does to maintain consistent state (i.e.
> receive and process the same input messages to produce same output, but not be allowed to talk
> to outputs if it sees the master's still here, but switch over to talking to them when the master
> bails out, effectively becoming master (and the other one when reboot is complete, will become shadow))

That is probably true, the BB's price range is in what I call
"afordable enough to get two". :-)


> Are you able to /abandon/ the dream of running NAS on it as well as control? :)

I guess I'm able to, if really necessary... but I will try hard not
to... :-)

I will not try to impose a hardware architecture on my system's users,
I just want to be able to identify one hardware architecture that can
work well and reliably at "reasonable" prices for users that don't
want to go to the trouble of determining thet for themselves. I did a
little "market research" among my beta-testers and found that 100+ EUR
for the control computer is considered expensive, although not
immediately a show stopper. But I don't call those prices "afordable
enough to get two", especially if, like the SheevaPlug (and similar
plug computers), the typical experienced MTBF is less than one year.

Anyway, I still have some time, the first commercially viable hardware
batch is being manufactured and will be ready to use only a couple of
months from now (if they work, of course!), so I guess I still have
about 6 months to get the server computer issues figured out.

Edwin van den Oetelaar

unread,
Mar 20, 2012, 1:46:27 PM3/20/12
to biffe...@googlegroups.com
although I am using Linux for everything, there are other ways to solve problems.
I recently found this project : http://eluabrain.blogspot.com/
It uses a low cost Cortex-M3 with a lua programming environment for eg. domotics.
I have not tested it, but seems to make sense in some way.


On Tue, Mar 20, 2012 at 6:37 PM, Rilhas <rogerio...@gmail.com> wrote:
> > I see your point, running it from FLASH (even if getting an updated
> > version from the hard disk) so that its performance were not affected
> > by the sluggish HDD response while it was being checked. I think that
> > is a good idea! How would I keep the swap file (which has to be big
> > and be kept on the hard disk) from affecting performance? Can I
> > somehow tell Linux that my domotics executable is not to be swapped
> > out?
[snip] 
Anyway, I still have some time, the first commercially viable hardware
batch is being manufactured and will be ready to use only a couple of
months from now (if they work, of course!), so I guess I still have
about 6 months to get the server computer issues figured out.

Andrew Scheller

unread,
Mar 20, 2012, 1:52:49 PM3/20/12
to biffe...@googlegroups.com
>> I can't tell if you're joking or if you're being serious... :-/
> Actually, I was joking. My cooking skills are not that great! :-)

LOL.

> This opinion of mine is mostly based on observing my work colleagues,
> and having watched some of their fights about Linux being better than
> Windows, or JAVA being better than C#, or red being better than green
> (footbal club colors), or whatever.

Pepsi vs. Coke, Tea vs. Coffee, Amiga vs. Atari, etc. etc.
Debian vs. Slackware, Gnome vs. KDE ;-)

I try to be a "right tool for the job" pragmatist.

> often with a good mood between adversaries. The discussions on Linux
> vs. <any other> tended to last less than an hour, but the underlying
> agressivness would last far longer into the following weeks, often
> hurting subsequent social behaviour between parties. I don't know why.
> I have a number of theories, but none explains all that I have seen.

Dunno, maybe feelings run deeper because people spend more time using
their OS than any particular programming language?

> explicitly said "there is absolutely no way I will ever want to learn
> how to do pudding, because I love it so much that I would endup
> puddiing myself to death". He insisted, he told me it was really easy,

Haha.

> But with Linux it is not like that, wizards really seem to be offended
> by those (like me) that don't really want to learn, the ones who just
> want to learn enough to use it as a tool. That is what I did when I

I have no problem with you learning to use Linux as a tool, as long as
you realise it's a more complex tool (especially in an embedded
environment like Bifferboard) than Windows running on a desktop PC ;)
When you said "I don't want to learn" I misunderstood you to mean "I
don't want to learn how to _use_ Linux. I just expect it to work, and
if it doesn't I'll just ask on the mailing list instead of searching
google first".

> I have nothing against Linux per se, I would like to be able to treat
> it just like I do Windows. After 20 years of using Windows I really
> know very little about Windows, and that is why I like it, it doesn't

After using it for 20 years, I suspect you've subconciously learnt
much more about it than you realise ;)

> Anyway, I would like to point out again I have never seen that in this
> forum (not in messages exchanged with me nor with messages that I've
> seen going by), and I think that is great! And Fabio tolerates me,

Pleased to hear it :) I think (hope) that as more and more people use
Linux, it gradually becomes less "elitist".
Ivory Towers. etc.

Lurch

Rilhas

unread,
Mar 20, 2012, 2:49:54 PM3/20/12
to Bifferboard

>
> Huh? No need to take it out of the closet and reflash the whole OS -
> just SCP a new binary to it and write it the the JFFS2 partition.

More magic!! ... I'll have to learn about that I guess, it is calling
out to my initial objective of running it from FLASH! Thanks! :-)

> OpenWRT is designed for embedded devices, so runs without swap (i.e.
> without a HDD), but this obviously means it can't run stuff that is
> "too big".

.... I see... maybe that is why Lukasz asked me if I could abandon the
NAS dream...

I'm not ready to go there just yet! Give me a couple more brick walls
to crash myself into and then I'll come back crying for help on
OpenWRT! :-)


>
> What if you had different "modes" programmed into the lightswitch
> microcontroller, and bifferboard just told the microcontroller which
> button-press-actions to trigger which light-switching-mode. Would be
> quicker than sending every single button press backwards and
> forwards... and does the 40 messages per second mean you're not
> allowed more than 40 lightswitches in your house ;-D

... well, not more than 40 lightswitches acting at the same time. But
in my house that would mean at least 20 persons fiddling with the
lights all at the same time! (the most I have close together is 2 lamp
switches). if that happens then there goes the 0.2 seconds response
time, but toggling 40 light switches at the same time would still
yield a response time of 0.1 seconds on the Bifferboard! .. unless if
the executable were swapped out due to some disk check hogging all the
RAM!

The "button-press-actions trigger which light-switching-mode" is a
possibility, but to prevent against hacking the microcontrollers will
have a "write protect mode" that will be activated once the user wnats
to get the system "in production". So if the user later changes his/
her mind the only way to reprogram the microcontrollers is to
physically get to them and short 2 pins while booting them. Also, the
conversion from duty-cycle to visible power is not linear (square and
sqare-rooted trigonometrical functions), so I didn't even tried to do
that in the microcontrollers. The only viable option I see would be to
load up the microcontrollers with pre-calculated tables for each lamp
type and interpolate, but I guess that would work. However, fallback
operation is already like that, but just for on/off, not for fancy
stuff (like changing LED lamp colours).

So that is not a solution for my domotics system, but I just intend to
provide hardware that you can use. The main challanges are
communications and a really tiny 240VAC->5VDC power supply that,
together, can fit behind a light switch, or a wall socket, that is
what usually gets people off automating their homes. The computer
software will be just enough for users to get started, and the
firmware is supposed to be good and need no change. So, if you want
you can reprogram everything, from computer to microcontrollers. But
my default firmware will have "fixed slave behaviour except during
emergencies" because I learned a long time ago that I will fail
miserably at trying to antecipate what people want to do with their
own domotics systems. Hence my attempts to transfer all control to a
central processor that can be easily programmed with conventional
programming languages.

>
> Just because you *can* run everything-and-the-kitchen-sink on a single
> bifferboard, doesn't necessarily mean it's a *good* idea... expecially
> if you're worried about response-times.
>

True, but trying it and succeeding was a good confidence-builder.

>
> Some would classify "files getting lost" as "fatal corruption".

LOL! Not me, fatal is not getting to boot. No way to recover from
that. It's much like losing a kidney, it can be a non-fatal problem,
the same with missing the spleen or one eye, but if you can't boot
everyday in the morning then you are as good as dead! That is what I
call fatal. In my book crippled>=dead.

>
> Just like NTFS, Ext3 and Ext4 are journalling filesystems. But
> obviously no system can ever claim to be 100% reliable. There's always
> lightning storms, solar flares, earthquakes, etc. ;-)

I disagree! Except for malfunctions or bit-rot (which I characterize
as malfunction, but it doesn't matter because not even backups can fix
bit-rot, mainly because they can also suffer from bit-rot) the read
and write operations required to support a filesystem can be designed
to never corrupt. That's what I want, castastrophes aside. You think
I'm ok with Ext3? Imagine power shortages and applications dying
unexpectedly... will Ext3 handle that in a way that can be compared to
NTFS?

>
> I suspect that as they're mechanical devices, harddrives probably
> dislike excess heat even more than the Bifferboard.

I read a report that Google issued a few years back. At the time
(around the time GoogleEarth launched) Google was the foremost
authority in harddisk testing, because they had the biggest collection
on the planet. Their report concluded that harddisks last longer if
you do not heed to common advices, like letting them cool, or reducing
the number of writes, or any of that jazz. On the contrary, it is much
more healthy to the disks to just use them intensively. I experienced
this first hand a few years before when a system that had been running
for 5 years non-stop was disconnected and the 4 machines were brought
back to our labs to do scheduled routine maintenance. The 30 minutes
the disks were disconnected caused irreversible damage to *ALL* of
them, and when they were back in our labs none booted or even spin. I
hammered a few just to understand what had happened and they
eventually began to spin, but the data was no longer recoverable.

I also have the same experience with computers that are on for long
periods of time. When you then turn them off for a day or two the
probability that they will not boot or that they won't work properly
from that point on is extremely high, much higher than if you didn't
turn them off at all. The heat cycles, especially the ones which are
very deep and very long, are most likely to cause severe irrecoverable
damage when interrupted.

The computer where I've been typing this message has a pair of RAID
disks which reach operating temperatures of 60ºC (I measured it), and
they haven't failed after 5 years.They both get equally hot, so I'm
trusting that is by design. Of course I have backups... :-)

So I don't think modern harddisks are as sensitive as one might think,
but don't bet your valuable data on that, it is just my opinion!

Fabio Silva

unread,
Mar 20, 2012, 2:57:50 PM3/20/12
to biffe...@googlegroups.com
It's time we all go SSD ;-)

I have a brand new spanking SSD Samsung 256 GB Hard Drive with Trim support and couldn't be happier with it without its moving parts, the fact that it's soldered to the motherboard and that it's giving me around 200 MBytes/sec of Read/Write rates!!!

Rilhas, Ext3 is considered as if not more reliable then NTFS in real world tests. Today Ext3 is used to power many real world production clusters and when even when more reliability/Functionality is required, companies are beginning to look at ZFS.

As for the argument you make about hard disks, couldn't agree more. I was responsible for a data center for 10 years and whenever I had to do a maintenance shutdown, I would always call the manufacturer of the array with standby disks because we would have one or two disks out of a batch of ten failing on us. This using RAID 5 at the time was not catastrophic, but always kept us "on edge".

Andrew Scheller

unread,
Mar 20, 2012, 3:21:52 PM3/20/12
to biffe...@googlegroups.com
>> Huh? No need to take it out of the closet and reflash the whole OS -
>> just SCP a new binary to it and write it the the JFFS2 partition.
> More magic!! ... I'll have to learn about that I guess, it is calling
> out to my initial objective of running it from FLASH! Thanks! :-)

http://en.wikipedia.org/wiki/Secure_copy

> The "button-press-actions trigger which light-switching-mode" is a
> possibility, but to prevent against hacking the microcontrollers will
> have a "write protect mode" that will be activated once the user wnats

"prevent against hacking" - huh? Is the domotics network not a
"closed-loop system"? Surely if an 'attacker' can write custom
firmware, they could just as easily hack into the
bifferboard->microcontroller messages?
If they DDoS your fridge, will the icecream melt? ;)

> together, can fit behind a light switch, or a wall socket, that is
> what usually gets people off automating their homes. The computer

Personally, I have no trouble using the "old fashioned" light
switches, where pressing the switch closes the circuit. Less than 1ms
response times ;-D

> own domotics systems. Hence my attempts to transfer all control to a
> central processor that can be easily programmed with conventional
> programming languages.

You're limiting your system to only being usable by programmers?

>> Just because you *can* run everything-and-the-kitchen-sink on a single
>> bifferboard, doesn't necessarily mean it's a *good* idea... expecially
>> if you're worried about response-times.
> True, but trying it and succeeding was a good confidence-builder.

Yeah, I guess it shows that you have "breathing room".

> to never corrupt. That's what I want, castastrophes aside. You think
> I'm ok with Ext3? Imagine power shortages and applications dying
> unexpectedly... will Ext3 handle that in a way that can be compared to
> NTFS?

No idea, that's not my area of expertise. You'll have to do some
research a.k.a. learning ;)

> on the planet. Their report concluded that harddisks last longer if
> you do not heed to common advices, like letting them cool, or reducing
> the number of writes, or any of that jazz. On the contrary, it is much
> more healthy to the disks to just use them intensively. I experienced

Okay, fair enough. Said by the man who tortures his hard disk by
spinning it down (during reboot) once a day... ;P

Lurch

Rilhas

unread,
Mar 20, 2012, 3:25:00 PM3/20/12
to Bifferboard
> Pepsi vs. Coke, Tea vs. Coffee, Amiga vs. Atari, etc. etc.
> Debian vs. Slackware, Gnome vs. KDE ;-)

Oh no... you didn't... you did... Amiga vs. Atari?? Let's not go
there! :-)

> Dunno, maybe feelings run deeper because people spend more time using
> their OS than any particular programming language?

... possibly... none of my theories go there, so it's a new
possibility for me to think about the next time I'm waiting for the
dentist. But I would guess that might not be it. In Portugal a lot of
people are really passionate about their football club, usually from
much earlier (fans start at 4 years of age), they live it every day
(keeping big posters on their cubicles and offices and what not), and
they defend that with a stronger conviction than possibly anything
else, even politics, or religion. And they fight over it, but usually
not to the point of affecting their social relationships, and usually
those discussions are easier to forget and move on.

Also, that still doesn't explain everything. I would say Windows users
spend at least as much time using the OS as their Linux counterparts,
but their response to users that don't care or to those who mouth off
is not the same. But that could have a deeper root, maybe Windows
attracts people that, like me, don't want to care about it, and with
that personality trait it may be easier to not care that much about
the discussion or not to take it so personally.

> After using it for 20 years, I suspect you've subconciously learnt
> much more about it than you realise ;)

I confess... I am one of the few people I know that knows that batch
files may take on parameters and that you can use them with %1 or %2
or %n. But try to keep it a secret because when I first mention that
to friends and co-workers it makes me look like a Windows wizard. :-)

Andrew Scheller

unread,
Mar 20, 2012, 3:35:40 PM3/20/12
to biffe...@googlegroups.com
> is not the same. But that could have a deeper root, maybe Windows
> attracts people that, like me, don't want to care about it, and with
> that personality trait it may be easier to not care that much about
> the discussion or not to take it so personally.

Or maybe because Windows comes pre-installed on 99% of computers,
means that Linux users by definition are more passionate because
they've deliberately chosen to install a different OS? *shrug*

> I confess... I am one of the few people I know that knows that batch
> files may take on parameters and that you can use them with %1 or %2

Isn't that more MS-DOS rather than Windows? ;) My favourite batch-file trick is
subst T: "%~dp0".

Anyway... that's probably enough off-topicness for one day! ;-)

Lurch

Fabio Silva

unread,
Mar 20, 2012, 3:41:23 PM3/20/12
to biffe...@googlegroups.com
Danger Will Robinson! You just brought up the "Amiga vs Atari" dispute to the greatest Amigaholic in the universe!!!

I apparently have a defect (or a bug). I tend to want to know how a OS works… Even a Mac ;-)

Rilhas

unread,
Mar 20, 2012, 3:45:26 PM3/20/12
to Bifferboard
>
> http://en.wikipedia.org/wiki/Secure_copy

LOL! I meant the JFFS2 thing, but I already Googled it. The magic is
gone (for now, waiting for the day I'll try to do it).

> "prevent against hacking" - huh? Is the domotics network not a
> "closed-loop system"? Surely if an 'attacker' can write custom
> firmware, they could just as easily hack into the
> bifferboard->microcontroller messages?

I guess the worst security breach that can happen is to reprogram the
microcontrollers. For those who thought that was a theoretical risk
Stuxnet made sure it is not.

So if some hacker gets control over the Bifferboard (or whatver PC may
be running the domotics software) they could potentially fiddle with
your lights and melt your icecream, but you could go to the closet,
replace the HDD with a fresh clean emergency pen disk, and disconnect
the BB from the internet. If the BB's FLASH got hacked you could also
replace the BB. That would be orders of magnitude less problematic
than dismantling light switches and blinds to get the microcontrollers
out and reprogram them (assuming you have the hardware to reprogram
microcontrollers).

So that could be the difference between say 1 hour downtime vs. 10
days and a big hassle. From my perspective that is the difference
between a viable domotics system and an unviable one.

Then you could take the time to find out how you got hacked. :-)


> If they DDoS your fridge, will the icecream melt? ;)

Not at first, so that you don't know you've been hacked. Only when you
have the 20 people over for your birthday dinner and you really need
the icecream to be cold... :-)

>
> Personally, I have no trouble using the "old fashioned" light
> switches, where pressing the switch closes the circuit. Less than 1ms
> response times ;-D

Ha. Buzzkill.

BTW: light switches usually bounce for 10ms to 20ms... but it still
trumps my 200ms. I'll think about it and get back to you when I have a
5ms version, then maybe you feel complled to buy! :-)

>
> You're limiting your system to only being usable by programmers?
>

Yes. Well, not really. The short answer is yes, but the long answer is
that programmers can then work for intaller companies, and those can
sell customized solutions for users who are not programmers. But
little old me should not have the presumption of being able to address
non-programmer markets directly.


> Okay, fair enough. Said by the man who tortures his hard disk by
> spinning it down (during reboot) once a day... ;P
>

LOL! I torture PIC's even more. One day I'll tell you the story of the
"underwater PIC" experiment, where I ran a betting pool at my
workplace about whether or not a LED flashing PIC and a battery would
work if submersed under water (no watertight compartment, just solder
the components and dump them in the sink full of water). That PIC
lived for 5 more days. Had it have children and grandchildren in those
5 days it would have a great story to tell them in front of the
fireplace. Anyway, it died of unrelated causes, a car ran over it when
I accidentally dropped on the road the day I was taking it to work
again to run a second betting pool.

Rilhas

unread,
Mar 20, 2012, 3:50:06 PM3/20/12
to Bifferboard
> although I am using Linux for everything, there are other ways to solve
> problems.
> I recently found this project :http://eluabrain.blogspot.com/
> It uses a low cost Cortex-M3 with a lua programming environment for eg.
> domotics.
> I have not tested it, but seems to make sense in some way.

Good one. I'll add it to my to do list, thanks!

Lukasz Sokol

unread,
Mar 21, 2012, 5:31:28 AM3/21/12
to biffe...@googlegroups.com
On 20/03/2012 17:37, Rilhas wrote:
[...]

> I guess I'm able to, if really necessary... but I will try hard not
> to... :-)
>
No but this is, contrary to what you may think, a simple tradeoff:
given the resources of a BB, and that you need close to 100% availability,
(whatever, automatic or manual fail switchover) running NAS on either one
is a bad idea, because you have to keep consistency between the two.

If you achieve _satisfying_ results trying to maintain consistency between
two identical systems without compromising on availability and control, and keeping
them synchronized file-wise, you may win a computer science Nobel prize :D

http://en.wikipedia.org/wiki/CAP_theorem

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

Extrapolating your setup to what that article says:
- consistency is when both bb's serve you the same HDD data AND perform same
control actions ALL the time
- Availability is when both files and control of your system is 99.99999999999%
available (given the time _power_ is connected to it as 100%, you* may not
afford more than 30s per year of downtime)
(* or your downstream partners!)
- partition tolerance matters because you need to guarantee all files be
the same on both hard drives at any given moment, regardless of what's
happening in control area without clashing with each other.

If you take out the 'AND FILES' part from the logic above, it may be easier to
paper-over the partition-tolerance as your control programs can handle the
rate of it today. (seamless software update on the two-bb system ? easy:
update the software on slave/shadow first, then make sure it functions and
force the master to give up power to shadow in a controlled manner, then
update software on it. Can be done over network without much hassle, and
given some version-awareness in bb's software, the master could give up power
by itself whenever it notices shadow being newer than him. )

All that is close to unachievable if you add the NAS function into equation -
for NAS, has its priorities a little different.

Not to mention, that even running rsync between the two mirror boards for
consistency in HDD files, takes not only processor time, but also transmission
time, and synchronizing can be time consuming, and affecting e.g. web server
responsiveness. And there is always time between rsync runs (executing rsync every
second is futile...) where one HDD can contain something else than the other.

HTH :)
L.

Andrew Scheller

unread,
Mar 21, 2012, 7:08:03 AM3/21/12
to biffe...@googlegroups.com
> be running the domotics software) they could potentially fiddle with
> your lights and melt your icecream, but you could go to the closet,

...or they could disable your CCTV, and unlock all your doors ;-P

> trumps my 200ms. I'll think about it and get back to you when I have a
> 5ms version, then maybe you feel complled to buy! :-)

Heh, okay.

> LOL! I torture PIC's even more. One day I'll tell you the story of the
> "underwater PIC" experiment, where I ran a betting pool at my
> workplace about whether or not a LED flashing PIC and a battery would
> work if submersed under water (no watertight compartment, just solder
> the components and dump them in the sink full of water). That PIC
> lived for 5 more days. Had it have children and grandchildren in those

I bet it wouldn't have worked so well if you'd put it in salty water!
And obviously the bigger the battery (voltage), the less chance of it
working.

Lurch

Rilhas

unread,
Mar 21, 2012, 7:46:02 AM3/21/12
to Bifferboard
>
> No but this is, contrary to what you may think, a simple tradeoff:
> given the resources of a BB, and that you need close to 100% availability,
> (whatever, automatic or manual fail switchover) running NAS on either one
> is a bad idea, because you have to keep consistency between the two.

I now understand what you mean, but for NAS redundancy is not a
requirement. Well, not from my perspective, but if users want their
NAS to be redundant that is their problem so let them find a solution
for that!

I'm totally comfortable with the idea that "redundancy" for my system
only covers essencial functions, like keeping the house functional,
lot locking you in or out, not keep you in the dark, not letting
strangers inside without your consent, and not melting your icecream.
All other features like watching photos on the TV or having the
webserver available don't need redundancy.

So I want to design it in a way that the system can do domotics and
NAS and webserver and SVN and the world-plus-a-pair-of-boots, but in
case something fails can still do domotics.

The example for redundancy that I mentioned before could include a
second BB as replacement for the first, a 4GB flash disk to replace
the 500GB harddisk (containing only the essential functionality), and
maybe a secondary (cheaper) radio. Things that would not be duplicated
include the Bluetooth range extender (250 EUR!), GSM modem (80 EUR),
etc., as many of these devices have a functional alternative, even if
not an exact replacement (still deciding on the GSM modem which is
used to notify of intrusion, an e-mail might not be a guaranteed
functional replacement, even if I program my phone to ring when one is
received).

But I think you are right. I use it for SVN server, which (from my
perspective) is even worse than NAS because redundancy would also be a
problem on the client computers and not just the NAS server. But, for
that, I rely on having "backup" copies checked out on all the
computers that can be commited if the SVN disk blows.

Andrew Scheller

unread,
Mar 21, 2012, 8:27:01 AM3/21/12
to biffe...@googlegroups.com
> But I think you are right. I use it for SVN server, which (from my
> perspective) is even worse than NAS because redundancy would also be a
> problem on the client computers and not just the NAS server. But, for
> that, I rely on having "backup" copies checked out on all the
> computers that can be commited if the SVN disk blows.

Does it also mean that if you're doing a big SVN update, all your
light-switches slow down? ;)

Lurch

Rilhas

unread,
Mar 21, 2012, 9:24:05 AM3/21/12
to Bifferboard
>
> Does it also mean that if you're doing a big SVN update, all your
> light-switches slow down? ;)
>

I sure hope not! And I hope the lamps don't start flickering slower
than 100Hz so that I don't start to see it! :-)

I've been able to get away with it without having to do anything. In
the university they tought me that, everything else being equal, an OS
should prioritize smaller tasks. Maybe that is what is happening here,
because no matter how busy the CPU seems to be the domotics server's
short message loop always shows decent response times. I guess that if
the executable were swapped out that would not be the case though.

In Windows this prioritization of small tasks doesn't seem to be
universally true (it randomly seems to be and not to be), so I do have
to take steps to prevent sluggishness. Since the laptop has only 512MB
of RAM and the video intrusion detection software that runs in it
takes about 700MB of that (!!) everything in it is usually swapped out
(even the Windows code that moves the mouse!!), so I had to tell
Windows not to swap out the domotics executable. Check.

Then, as the laptop's CPU is 1GHz and the intrusion detection software
uses about 4GHz of that (!!) I had to raise the priority of the
domotics process. Check.

After that I had to tweak the domotics thread's priorities, because
raising the process priority somehow unballances the threads'
scheduling. So the web crawler thread's priority (and that of other
non time critical threads) has to be lowered or, alternatively, the
message loop thread's priority has to be raised. Check.

As this is all configurable through text files it is not a real
problem to tune it, as once you find a good ballance then it usually
holds no matter how many RAR compressions you decide to run in the
same machine.

Since I didn't have any problem in Linux the process and swap control
parts of the multitasking module have not yet been ported, but I guess
I should eventually do it. Since for now the "production" machine is
still Windows and I only run the software on the Bifferboard in
shorter test runs, porting the rest of the module has been pushed back
in the list. Although, as mentioned before, I may be unable to tell
Linux not to swap out the domotics process, and that doesn't leave me
very confident...

Andrew Scheller

unread,
Mar 21, 2012, 9:38:20 AM3/21/12
to biffe...@googlegroups.com
> to take steps to prevent sluggishness. Since the laptop has only 512MB
> of RAM and the video intrusion detection software that runs in it
> takes about 700MB of that (!!) everything in it is usually swapped out
> (even the Windows code that moves the mouse!!), so I had to tell
> Windows not to swap out the domotics executable. Check.
[snip] [snip] [snip]

Or you could just take the easy way out and buy a memory upgrade ;)

Lurch

Rilhas

unread,
Mar 21, 2012, 9:57:55 AM3/21/12
to Bifferboard
>
> Or you could just take the easy way out and buy a memory upgrade ;)
>

I'm much too cheap for that! :-)

... and the Windows laptop is on its way to laptop heaven. I expected
it to die about 4 months ago, because the fan started making strange
noises 6 months ago. Also the fan no longer stops. I disassembled the
laptop to see if I could get to the fan, but no joy (I only had 15
minutes because like I mentioned before after being on for 4 years non-
stop I fear turning it off for longer than that would be, although
mercyful, fatal.

Since this thread is blatantly off-topic (and has been so for quite a
while) I take this oportunity to share with you that I will probably
be leaving Windows in the near future. What made it great for me all
these years (not having to learn about it, powerful and consistent
keyboard access to everything in the OS and nearly all applications)
seems to have its days numbered with the arrival of Windows 8. Maybe
they keep some of the old productivity mechanisms, but many have
already been abandoned in Vista and 7, so I don't keep my hopes up.

So, investing 20 euros on another 512MB of RAM is not a good bet for
the future because Windows 8 is comming. :-)

Andrew Scheller

unread,
Mar 21, 2012, 10:54:55 AM3/21/12
to biffe...@googlegroups.com
> while) I take this oportunity to share with you that I will probably
> be leaving Windows in the near future. What made it great for me all

I guess if you're leaving Windows, and you don't want to learn how to
use Linux, your only option left is MacOSX and its overpriced hardware
;-)

Lurch

Rilhas

unread,
Mar 21, 2012, 10:59:33 AM3/21/12
to Bifferboard
>
> I guess if you're leaving Windows, and you don't want to learn how to
> use Linux, your only option left is MacOSX and its overpriced hardware
> ;-)
>

... yes, my future looks grim...

I guess I'll fold and give that Ubuntu thing another try. Maybe it
doesn't expect too much from me and I don't need to sudo a lot! :-)

rolf

unread,
Mar 21, 2012, 11:22:51 AM3/21/12
to Bifferboard
On 21 Mrz., 16:54, Andrew Scheller <ya...@loowis.durge.org> wrote:

> I guess if you're leaving Windows, and you don't want to learn how to
> use Linux, your only option left is MacOSX and its overpriced hardware

I think that's mythology meanwhile. First, MacOSX is Unix as well.
Then, about a year ago I was just at this point, thinking to buy a Mac
for my wife (not for me), just because I was tired to try to help her
with this Windows problems. Before I did I asked some experienced Mac-
users, who knew Linux as well. Their answer: MacOS not the hundred
percent easy use and reliable OS anymore, it's comparable with Linux
now.

Rolf

Andrew Scheller

unread,
Mar 21, 2012, 11:36:53 AM3/21/12
to biffe...@googlegroups.com
> with this Windows problems. Before I did I asked some experienced Mac-
> users, who knew Linux as well. Their answer: MacOS not the hundred
> percent easy use and reliable OS anymore, it's comparable with Linux
> now.

...with the caveat that Linux doesn't have swanky "Genius Bars" in
city centres staffed by oh-so-fashionable geeks ;-)

Lurch

Andrew Elwell

unread,
Mar 21, 2012, 1:48:14 PM3/21/12
to biffe...@googlegroups.com


STEP 1: swanky Linux "Genius Bars" in city centres staffed by oh-so-fashionable geeks ;-)

STEP 2: ???

STEP 3: Profit

Nope, that one doesn't work either. Back to $dayjob then.

Rilhas

unread,
Apr 9, 2012, 8:20:09 AM4/9/12
to Bifferboard
Hi all,

The "fstab 0-0" seemed to work, but unfortunatelly this morning I was
greeted with the "25 mounts since last check" again. So I used the
"tune2fs -c 0 -i 0" to shut it up and it seems to have worked. So I
guess one needs to use both, tune2fs to disable periodic checks and
fstab to disable all others. That assuming that it is now permanently
fixed, of course.

... *sigh*

Andrew Scheller

unread,
Apr 9, 2012, 2:54:28 PM4/9/12
to biffe...@googlegroups.com
> guess one needs to use both, tune2fs to disable periodic checks and
> fstab to disable all others. That assuming that it is now permanently

http://linux.die.net/man/8/tune2fs
http://linux.die.net/man/5/fstab

;-)

Lurch

Rilhas

unread,
Apr 9, 2012, 5:19:20 PM4/9/12
to Bifferboard
I have read those links and there is where I was mislead into thinking
fstab would solve it. To quote fstab man page:

"[...] If the sixth field is not present or zero, a value of zero is
returned and fsck will assume that the filesystem does not need to be
checked. "

So, naturally, I thought setting the 6th field to zero would solve
*ALL* possible filesystem checking needs. Apparently that is not true
because the "25 mounts since last check" came back to haunt me. I read
the tune2fs man page to see what checks might be performed, and only
got the -c and -i (I decided to leave -e alone for now).

I'm still convinced something is missing, like some obscure case for
when some bits on whichever file's time stamp are all even and a check
is thus needed or something like that. So I'm waiting for the next
"stupid problem". If something like this happens again I think I will
replace the "fsck" executable with a stub that just outputs "Don't
worry, be happy!". That is what I would call a definitive
solution! :-)
Reply all
Reply to author
Forward
0 new messages