Beaglebone crashing during thunderstorms!

[Wally Bkg]

It happened again, my Beaglebone White crashed as a thunderstorm moved through the area. It is on a UPS, the same UPS as my router ASUS RT-AC56 and Raspberry Pi2, neither of which glitched in any way.

It does have a lot of interface wiring hooked up but these are very thoroughly EMP protected with Transorbs and have been in use for 20+ years surviving tropical storm Allison, Hurricane Ike, and many other national news worth storms in the Houston area -- this was a pitiful storm by our standards never even making the lights noticeably flicker, although the UPSes did "beep" once. This protection was added in a rebuild after EMP from Hurricane Alicia wiped out the initial system (pre-IBMPC CMOS logic).

Why is the BBW far more sensitive to this than the RPi2 using similar 2A+ wall-wart power supplies plugged into the same UPS?

My initial "work-around" would be a watchdog running on the RPi2 to active a normally closed relay to interrupt the BBW power supply so it'd restart after after the BBW dies. I've already have in place network monitoring of the BBW status and a "heartbeat" to detect when it stops. But I'd rather the BBW didn't crash :(

[Gerald Coley]

Well, I don't know where to start. A detailed diagram of the entire system hookup would help.

Gerald

--
For more options, visit http://beagleboard.org/discuss
---
You received this message because you are subscribed to the Google Groups "BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beagleboard...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/9305eb4a-be7e-4c4e-8985-404dc92fcc12%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Gerald

 

ger...@beagleboard.org

http://beagleboard.org/

gco...@emprodesign.com

[Graham]

There is a whole "hygiene" relative to grounding and ESD protection.

Is it in a metal box?

Is the metal box grounded?

Are all of your power supplies grounded? 

Did you actually measure it?

(Most U.S. wall warts don't connect to safety ground, so the whole system is floating, and anything can happen.)

Are there ground referenced ESD clamps on all I/O leaving the metal box?

Is the bench or table it is sitting on grounded? ESD mat?

Without good grounding and protection "hygiene" all external wires, including the Ethernet cables, the external wires all act as antennas to pick up external noise and signals, including induced voltages from lightning.

A wall-wart without a safety-ground pin connection is the worst thing you can run something from, from the aspect of ESD protection.

--- Graham

==

[evilwulfie]

I solve these issues by using opto isolators on anything that leaves the box the BBB is mounted in.

To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/cae943c7-6211-4796-9037-f37dcbe56cf8%40googlegroups.com.

[Wally Bkg]

"Well, I don't know where to start. A detailed diagram of the entire system hookup would help.

Gerald"

Being in active use for ~25 years and hooked to a variety of computers as the years have passed, its never been a problem, but the best I could do would be a scan of the diagrams I've built it from. Its very repetitive, four basic interface circuit designs repeated 2, 4, 8, 16 times. But I may get desperate enough to do this, eventually.

But it acted as if the power button had been held down and just shut off, and there was minimal disturbance to lights (mix of incandescent, CF, and LED) and the various USPes around the house, so I'm thinking maybe EMP sensitivity in the power controller chip. Its an old house (built circa 1950), but this outlet is one that I ran a 12ga stranded wire straight from the outlet to the grounding rod at the service entrance when we first moved in to give me a good ground back in the days I did a lot of hardware work -- remember the days of wire-wrapped S-100 bus prototypes :)

OTOH, as I've upgraded the security cameras to HD I've reused the SD camera wiring to add additional PIR motion detectors, and I just discovered (after posting this thread) I'd one added PIR unintentionally powered from a different UPS (but they are all on the same breaker and safety ground connection), so I'll see if it happens again. This had been the case for the first thunderstorm crash a few months ago as well.

[Gerald Coley]

So, on this crash....Does it basically reset or just stop? How do you recover?

Gerald

--
For more options, visit http://beagleboard.org/discuss
---
You received this message because you are subscribed to the Google Groups "BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beagleboard...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/6cc15654-ade8-4aea-821b-c7bed75f5575%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[William Hermans]

Wally, why don't you put shielding around your beaglebone, and see if it happens next time.

We've seen very similar reset situations here with PTP WIFI routers ( WDS ), from very close lightning strikes we get here every year this time of year. Yet, none of our beaglebone's have been affected.

So what's the difference between you, and us ? We live inside a 80' x 100' steel building. In other words, all our equipment is shielded, except the PTP routers which are outside of the building . . .

To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/CAHK_S%2Be95AW5w7o5gwcyvnULgy_3cvVqOVMOmj30g-P5tJF3Ag%40mail.gmail.com.

[William Hermans]

But yeah have a look at this: https://plus.google.com/106867156582775247949/posts/gFPzDcwyWCs The second image on that post is a really good close up, of what happens to equipment out here once in a while ;)

[Graham Haddock]

The PMIC in the BeagleBone will go into shutdown, if the DC supply voltage goes below 4.5 Volts, or above 5.5 Volts, even for a few milliseconds.

In use it should be held between 4.75 and 5.25, so you have some margin.

If the "lights flickered" then the power supply for the BBB did not have to shut down or loose power, just loose regulation enough to go below 4.5 or above 5.5.

The BBB is more sensitive than a lot of devices in that respect.

--- Graham

==

You received this message because you are subscribed to a topic in the Google Groups "BeagleBoard" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/beagleboard/l8hri0SzSKE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to beagleboard...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/CALHSORrnE%2BbWWhr6soH1S6XOecSCHshoysY9aOO_AajN_S-TEw%40mail.gmail.com.

[Wally Bkg]

Thanks for the info about the input voltage sensitivity of the Beaglebone, good chance this is the real issue, especially as there was a good chance for "ground bounce" between different parts of the system from my mistake in powering one of the PIR sensors off the "wrong" UPS. Time will tell. If it happens again now that I've corrected that mistake, I'll look into powering the BB with a better supply, and/or look into adding the backup battery option.

Due to the location, shielding the BB in a metal box is way too invasive to try. Since the RPi2 has had no issues either time I consider the evidence that shielding would help to be shaky at best.

I got it booted again by unplugging the BB wal-wart 5V supply and plugging it back in. As a last resort I can add a watchdog and relay to interrupt the BB power and re-apply it after ~30 seconds.

[William Hermans]

Thanks for the info about the input voltage sensitivity of the Beaglebone, good chance this is the real issue, especially as there was a good chance for "ground bounce" between different parts of the system from my mistake in powering one of the PIR sensors off the "wrong" UPS.  Time will tell.  If it happens again now that I've corrected that mistake, I'll look into powering the BB with a better supply, and/or look into adding the backup battery option.

If the cause is in fact ESD somehow triggering the Beaglebone reset. Adding a battery won't help unless you remove the acpid package.  However I will add that pressing the power button, and input voltage going away both fire the same interrupt. So if it was power momentarily dipping below the 5v threshold. Adding a battery should fix that. Do note, though unless you want your board to power down immediately, you'll want to remove the acpid package . . . 

So, just to be 100% clear. the acpid package ( service ) triggers both button press, and 5v input voltage shutdowns. Remove it, and neither should be triggered afterwards.

--
For more options, visit http://beagleboard.org/discuss
---
You received this message because you are subscribed to the Google Groups "BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beagleboard...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/023e9e06-2c70-4980-a197-ed9435c7a90c%40googlegroups.com.

[Wally Bkg]

On Sunday, July 3, 2016 at 11:13:12 AM UTC-5, William Hermans wrote:

Thanks for the info about the input voltage sensitivity of the Beaglebone, good chance this is the real issue, especially as there was a good chance for "ground bounce" between different parts of the system from my mistake in powering one of the PIR sensors off the "wrong" UPS.  Time will tell.  If it happens again now that I've corrected that mistake, I'll look into powering the BB with a better supply, and/or look into adding the backup battery option.

If the cause is in fact ESD somehow triggering the Beaglebone reset. Adding a battery won't help unless you remove the acpid package.  However I will add that pressing the power button, and input voltage going away both fire the same interrupt. So if it was power momentarily dipping below the 5v threshold. Adding a battery should fix that. Do note, though unless you want your board to power down immediately, you'll want to remove the acpid package . . . 

So, just to be 100% clear. the acpid package ( service ) triggers both button press, and 5v input voltage shutdowns. Remove it, and neither should be triggered afterwards.

If it happens again, meaning that my power distribution error was not the reason, I'll look into removing the acpid package, as 24/7 operation is the goal, normally it just gets rebooted after software upgrades (my code continues to evolve) via sudo reboot.  I'd only want a real shutdown if I was to modify the wiring -- something that is pretty much frozen now that its been mounted in its "permanent" home.  Removing acpid package would be the easiest, followed by a better 5V supply.   The backup battery is the least appealing as it'll require removing it from its home :(

 

These infrequent events are hard to track down,  adding the watchdog to auto power cycle might actually be easier, and more robust in the long run, than adding the battery now that I've looked into how to do that.

[John Syne]

You want to be very careful doing this. The BBB has an issue where the 3v3 voltage rail should not be more than 2v more than the 1v8 rail. When you remove power suddenly, the 1v8 rail can drop faster than the 3v3 rail and damage the processor. If you look at the Octavo reference design, you will see additional circuitry to prevents this from happening. Best to shut the power down slowly before restarting, or just toggle the reset line.

http://octavosystems.com/octavo_products/osd335x/

Regards,
John

>
> --
> For more options, visit http://beagleboard.org/discuss
> ---
> You received this message because you are subscribed to the Google Groups "BeagleBoard" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to beagleboard...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/023e9e06-2c70-4980-a197-ed9435c7a90c%40googlegroups.com.

[John Syne]

It won’t be EMC related as the BBB PCB layout is pretty good and isn’t susceptible to lightning. My guess would be related to power flowing across the board. To prevent this, don’t use more than one ground connection between the BBB and your other circuits. Preferably, use the pins closest to the 5V power socket. Use opto isolators were ever possible. To test this, use a Taser gun approximately 1 meter (3 feet) away. 

Regards,

John

On Jul 3, 2016, at 9:12 AM, William Hermans <yyr...@gmail.com> wrote:

Thanks for the info about the input voltage sensitivity of the Beaglebone, good chance this is the real issue, especially as there was a good chance for "ground bounce" between different parts of the system from my mistake in powering one of the PIR sensors off the "wrong" UPS.  Time will tell.  If it happens again now that I've corrected that mistake, I'll look into powering the BB with a better supply, and/or look into adding the backup battery option.

If the cause is in fact ESD somehow triggering the Beaglebone reset. Adding a battery won't help unless you remove the acpid package.  However I will add that pressing the power button, and input voltage going away both fire the same interrupt. So if it was power momentarily dipping below the 5v threshold. Adding a battery should fix that. Do note, though unless you want your board to power down immediately, you'll want to remove the acpid package . . . 

So, just to be 100% clear. the acpid package ( service ) triggers both button press, and 5v input voltage shutdowns. Remove it, and neither should be triggered afterwards.

On Sun, Jul 3, 2016 at 8:13 AM, Wally Bkg <wb666...@gmail.com> wrote:

Thanks for the info about the input voltage sensitivity of the Beaglebone, good chance this is the real issue, especially as there was a good chance for "ground bounce" between different parts of the system from my mistake in powering one of the PIR sensors off the "wrong" UPS.  Time will tell.  If it happens again now that I've corrected that mistake, I'll look into powering the BB with a better supply, and/or look into adding the backup battery option.

Due to the location, shielding the BB in a metal box is way too invasive to try.  Since the RPi2 has had no issues either time I consider the evidence that shielding would help to be shaky at best.

I got it booted again by unplugging the BB wal-wart 5V supply and plugging it back in.  As a last resort I can add a watchdog and relay to interrupt the BB power and re-apply it after ~30 seconds.

--
For more options, visit http://beagleboard.org/discuss
---
You received this message because you are subscribed to the Google Groups "BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beagleboard...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/023e9e06-2c70-4980-a197-ed9435c7a90c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
For more options, visit http://beagleboard.org/discuss
---
You received this message because you are subscribed to the Google Groups "BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email to beagleboard...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/beagleboard/CALHSORrdSAFGvf%2Bzkf0K7mkR-G4rWWc1-4rvArjU%3DeOHDJwP9A%40mail.gmail.com.