BeagleBone A5 freezes on login

[Kristian]

Hello,

I have a couple of BeagleBone rev. A5 running the latest available kernel downloaded from rcn-ee.net (psp10 from 15. May) and Debian Squeeze. On most of the devices, everything works as it should. However, some of the machines freezes directly after login. What happens is that as soon as I log in, the machine freezes and the I/O leds on the board switch off. The power led is still shining green. I have tried with several different memory cards, power from USB and 5V and with and without my USB devices connected. Also, when I look at the boot log, I can't see anything that is out of the ordinary, the boot seems to be sucessful. There is no error outpt to screen nor serial port.

Has anyone experienced anything similar or have any tips on how I can progress in debugging this? I have uploaded the bootlog to: http://pastebin.com/mfApp8sA 

-Kristian 

[Robert Nelson]

Yes, i've had reports of this freeze issue, but i've been unable to
reproduce that issue in the lab with any of my A2/A3/A5 boards.. For
the time being you can use the angstrom uImage kernel and modules just
fine on the debian rootfs..

If you'd like to help debug the issue, the source/build tree is available here:

https://github.com/RobertCNelson/linux-dev/tree/am33x-v3.2

(hint: it's probally a config difference between my tree and angstrom's..)

Regards,

--
Robert Nelson
http://www.rcn-ee.com/

[Koen Kooi]

Don't switch frequency with cpufreq, it oopses the kernel nowadays in softidle and irqs. Maybe the irq bank fix has some unintended side effects :(

[Robert Nelson]

Any chance is that related to the same bug that some boards suspend,
but others work fine.. (and i just happened to luckily get the good
bone's?) ;)

[Koen Kooi]

Suspend/resume is a silicon issue last I heard. Kinda like 1GHz on xM, select boards work, so there is no problem....

[Kristian]

Thanks for the tip. Where can I find the angstrom uImage/modules for the latest kernel? I have had lots of problems with Usb in kernels earlier than psp7, what would be the angstrom equivalent of that kernel?

When I have some spare time, I will start experimenting with the configs. I see this behavior on multiple boards, but have not been able to find a systematic difference.

Thanks again for the help,
Kristian

[Kristian]

Also, if anyone has any suggestions for config options to enable/disable, please let me know.

-Kristian

[Robert Nelson]

On Wed, May 16, 2012 at 11:41 AM, Kristian <kristian...@gmail.com> wrote:
> Also, if anyone has any suggestions for config options to enable/disable, please let me know.

With koen's suggestion i just tweaked this..

https://github.com/RobertCNelson/linux-dev/commit/bf451d395ff25352a37573b1b3f73a4d3b6fdb0b

I'll have a uImage/modules ready for download/testing in 10mins, which
i'd like to have you test as your board is showing the issues..

[Koen Kooi]

Op 16 mei 2012, om 18:44 heeft Robert Nelson het volgende geschreven:

> On Wed, May 16, 2012 at 11:41 AM, Kristian <kristian...@gmail.com> wrote:
>> Also, if anyone has any suggestions for config options to enable/disable, please let me know.
>
> With koen's suggestion i just tweaked this..
>
> https://github.com/RobertCNelson/linux-dev/commit/bf451d395ff25352a37573b1b3f73a4d3b6fdb0b

You need the reverse diff of that :) USERSPACE is the safest bet, provided nothing in userspace tries to touch cpufreq.

[Robert Nelson]

But we can't reverse that, as that's the way it was in his image by
default (using userspace cpufreq) and since the user space tool
cpufreq is installed by default.. :) So either have user remove the
cpufreq tools, or just force performance...

[Robert Nelson]

On Wed, May 16, 2012 at 11:41 AM, Kristian <kristian...@gmail.com> wrote:

> Also, if anyone has any suggestions for config options to enable/disable, please let me know.
>

Can you give this a shot:

http://rcn-ee.homeip.net:81/testing/beaglebone/3.2-cpufreq/

Just cp the uImage file as uImage to the boot partition and extract
the modules as root to the rootfs partition..

[Anunakin]

I have issues with Angstron, now using Ubuntu r8 11.10 with a 256MB swap files, that works nice ...

[Kristian Evensen]

Thanks. I only have remote access to the machines the rest of today
and tomorrow, and the machine that freezes on boot has crashed (there
is the successful boot now and then). I will try this first thing
Friday morning.

-Kristian

[Rob Fowler]

I tried this kernel on my R4, same problem

done.

Begin: Running /scripts/init-bottom ... done.

[  240.780506] INFO: task init:1 blocked for more than 120 seconds.

[  240.786841] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.

[  240.795101] init            D c04d4c7c     0     1      0 0x00000001

[  240.801818] Backtrace:

[  240.804429] [<c04d4890>] (__schedule+0x0/0x43c) from [<c04d4de8>] (schedule+0x78/0x7c)

[  240.812795] [<c04d4d70>] (schedule+0x0/0x7c) from [<c04d5128>] (schedule_timeout+0x20/0x16c)

[  240.821705] [<c04d5108>] (schedule_timeout+0x0/0x16c) from [<c04d4648>] (wait_for_common+0xc8/0x154)

[  240.831328]  r6:00000002 r5:cf837abc r4:7fffffff

[  240.836216] [<c04d4580>] (wait_for_common+0x0/0x154) from [<c04d46ec>] (wait_for_completion+0x18/0x1c)

[  240.846042] [<c04d46d4>] (wait_for_completion+0x0/0x1c) from [<c004beb8>] (flush_work+0x30/0x3c)

[  240.855322] [<c004be88>] (flush_work+0x0/0x3c) from [<c02e79fc>] (tty_flush_to_ldisc+0x14/0x18)

[  240.864503] [<c02e79e8>] (tty_flush_to_ldisc+0x0/0x18) from [<c02e2b2c>] (n_tty_poll+0x7c/0x180)

[  240.873790] [<c02e2ab0>] (n_tty_poll+0x0/0x180) from [<c02e021c>] (tty_poll+0x68/0x88)

[  240.882134]  r7:00000000 r6:ced16ac0 r5:ced2c000 r4:cf2be980

[  240.888131] [<c02e01b4>] (tty_poll+0x0/0x88) from [<c00cbf24>] (do_select+0x2b8/0x460)

[  240.896475]  r7:00000000 r6:00000002 r5:00000200 r4:00000400

[  240.902481] [<c00cbc6c>] (do_select+0x0/0x460) from [<c00cc310>] (core_sys_select+0x244/0x308)

[  240.911573] [<c00cc0cc>] (core_sys_select+0x0/0x308) from [<c00cc4b0>] (sys_select+0xdc/0x10c)

[Robert Nelson]

On Thu, May 17, 2012 at 6:16 AM, Rob Fowler <mia...@gmail.com> wrote:
>
> I tried this kernel on my R4, same problem
> done.
> Begin: Running /scripts/init-bottom ... done.

Rob, can you be a little more specific, i there's about 3 revision
floating around since yesterday..

With another users, this one seems to be solving his problems:

http://rcn-ee.homeip.net:81/testing/beaglebone/3.2-cpufreq-disable-oppturbo/

[Rob Fowler]

I mean this pre-built one one I just tested:  http://rcn-ee.homeip.net:81/testing/beaglebone/3.2-cpufreq/ 

(I also tried one I  cooked up from your instructions before that).

I'll try the 3.2-cpufreq-disable-oppturbo   one now.

On Thursday, 17 May 2012 21:34:13 UTC+10, RobertCNelson wrote:

[Robert Nelson]

On Thu, May 17, 2012 at 6:49 AM, Rob Fowler <mia...@gmail.com> wrote:
> I mean this pre-built one one I just tested:
> http://rcn-ee.homeip.net:81/testing/beaglebone/3.2-cpufreq/
> (I also tried one I  cooked up from your instructions before that).
> I'll try the 3.2-cpufreq-disable-oppturbo   one now.

Okay, thank's for confirming that..

Have you found a way to quickly replicate the error (certain program,
certain load, etc).. On my A3 it happened once yesterday, but i've
been unable to replicate it since..

[Rob Fowler]

Just booted [    0.000000] Linux version 3.2.17-psp10.3 (voodoo@hera) (gcc version 4.6.3 (Linaro GCC 4.6-2012.03) ) #3 Wed May 16 18:26:45 CDT 2012

from the  3.2-cpufreq-disable-oppturbo pre built directory and same problem.

Maybe I am chasing a furfie here, mine does it every time with arm ubuntu. It boots the old angstrom fine.

Does the hungtimeout stack back trace mean anything? It's the same every time.

[Sid Boyce]

I am having the same problems on an A4 board.
All was fine from the original 12.04 Alpha install and upgrades up until a recent "aptitude full-upgrade" when I ran into the problem.
All attempts at new installs and substitutions of 3.2.17-psp10.1-modules.tar.gz,  3.2.17-psp10.1.uImage,  linux-image-3.2.17-psp10.2_1.0cross_armel.deb result in the same problem.
Regards
Sid.

-- To join: http://beagleboard.org/discuss
To unsubscribe from this group, send email to:
beagleboard...@googlegroups.com
Frequently asked questions: http://beagleboard.org/faq

-- 
Sid Boyce ... Hamradio License G3VBV, Licensed Private Pilot
Emeritus IBM/Amdahl Mainframes and Sun/Fujitsu Servers Tech Support
Senior Staff Specialist, Cricket Coach
Microsoft Windows Free Zone - Linux used for all Computing Tasks

[Robert Nelson]

On Thu, May 17, 2012 at 7:29 AM, Sid Boyce <sbo...@blueyonder.co.uk> wrote:
> I am having the same problems on an A4 board.
> All was fine from the original 12.04 Alpha install and upgrades up until a
> recent "aptitude full-upgrade" when I ran into the problem.
> All attempts at new installs and substitutions of
> 3.2.17-psp10.1-modules.tar.gz,  3.2.17-psp10.1.uImage,
> linux-image-3.2.17-psp10.2_1.0cross_armel.deb result in the same problem.

Starting to loose hair in this one..

Images to test..
http://rcn-ee.homeip.net:81/testing/beaglebone/3.2-cpufreq-more/

Thought it might be sd card related, so tested 7 different micro sd
cards, along with the an A2 (main patch in that kernel change), a3 and
a5 beaglebone.. I got some more boards at home to try and more micro
sd cards.. But i can't get these boards to have that error..

[Robert Nelson]

Humm, so it took awhile to finally trigger one, but then it only
occurred since my nfs server had been rebooted...

[ 28.945678] eth0: no IPv6 routers present
[87216.428833] nfs: server 192.168.1.95 not responding, timed out
[87360.618927] INFO: task expect:27318 blocked for more than 120 seconds.
[87360.625885] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[87360.634246] expect D c04e4c50 0 27318 21265 0x00000000
[87360.641052] Backtrace:
[87360.643707] [<c04e4864>] (__schedule+0x0/0x43c) from [<c04e4dbc>]
(schedule+0x78/0x7c)
[87360.652160] [<c04e4d44>] (schedule+0x0/0x7c) from [<c04e4e58>]
(io_schedule+0x98/0x108)
[87360.660736] [<c04e4dc0>] (io_schedule+0x0/0x108) from [<c0089660>]
(sleep_on_page+0x10/0x18)
[87360.669738] r4:cec89dec r3:0000000d
[87360.673583] [<c0089650>] (sleep_on_page+0x0/0x18) from [<c04e5328>]
(__wait_on_bit+0x5c/0xa8)
[87360.682678] [<c04e52cc>] (__wait_on_bit+0x0/0xa8) from [<c00897fc>]
(wait_on_page_bit+0x98/0xa8)
[87360.692047] [<c0089764>] (wait_on_page_bit+0x0/0xa8) from
[<c00898cc>] (filemap_fdatawait_range+0x70/0x148)
[87360.702423] r5:ffffffff r4:c14624b0
[87360.706268] [<c008985c>] (filemap_fdatawait_range+0x0/0x148) from
[<c008a7c4>] (filemap_write_and_wait_range+0x5c/0x70)
[87360.717773] [<c008a768>] (filemap_write_and_wait_range+0x0/0x70)
from [<c0189e64>] (nfs_file_fsync+0x68/0x13c)
[87360.728424] r9:00000000 r8:00000000 r7:00000006 r6:00000000 r5:c5d67900

[Kristian]

Images to test..
http://rcn-ee.homeip.net:81/testing/beaglebone/3.2-cpufreq-more/

I have tested this image on my BeagleBone now, using the same memory cards as earlier. The device crashes during boot, seemingly right before the loging prompt would be shown. There is unfortunatly no crash report written to the serial port.

Wrt. applications, the crash seems mostly to occur when I log in. It does not matter if I log in through SSH or the serial port. What is strange is that my start-up scripts most of the time run correctly, the crash seems to be triggered by me logging in to the device.

-Kristian 

[Kristian]

I have experimented with the Angstrom kernel now, and I see the same behavior. I replaced uImage and modules with the ones from here [1], and after a seemingly successful boot, the device crashes.

-Kristian

[1] - http://downloads.angstrom-distribution.org/demo/beaglebone/

[Rob Fowler]

I tried this  /3.2-cpufreq-more/  version and have the same crash but now I am looking at the backtrace it is different to yours at the top but still ends up stalling when it 's attempting to schedule io. Mine seems to be locking up scheduling tty io which I assume is eventually going into the USB serial device.

Does anyone know if the wait for completion and schedule encapsulates some sort of interrupt setup and callback? Interrupts not working maybe?

[Kristian]

I did some more experimentation, with a new kernel built today (git describe gives  3.2-psp9-2-gd2cef3a). I thought I had tried the different power inputs before, but maybe I remember wrong.

If I boot using USB, then everything works and the device seems stable. I am not sure that the crash will not occur, but the machine has been up for the last couple of hours. On a second machine where the crash-problem also occurs, I have tried multiple reboots (USB-powered) and I have been able to log in every time (both SSH and serial).

If I, on the other hand, boot using the 5V, then the machine crashes as described earlier. The crash either happens during the boot sequence or when I try to log in. This also happens on the other machine with the crash-problem.

-Kristian

[Sid Boyce]

A curious thing happened tonight.
I had to power off the PC so there was no serial console connection and
suddenly I noticed the BeagleBone had booted and there was a login
prompt on the 7" Liliput USB screen.

I am logged in and installing packages.
tindog:/home/lancelot # ssh -X bone
The authenticity of host 'bone (192.168.10.19)' can't be established.
ECDSA key fingerprint is 0b:d0:ca:10:2c:26:9e:36:bc:02:ce:e1:b7:54:0a:3f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'bone,192.168.10.19' (ECDSA) to the list of
known hosts.
root@bone's password:
Welcome to Ubuntu 11.10 (GNU/Linux 3.2.17-psp10.1 armv7l)

* Documentation: https://help.ubuntu.com/
Last login: Sat May 19 17:41:36 2012
/usr/bin/xauth: file /root/.Xauthority does not exist
root@omap:~# uname -r
3.2.17-psp10.1
root@omap:~#
Regards
Sid.

[Rob Fowler]

I have now tried every ubuntu version and release from this page http://elinux.org/BeagleBoardUbuntu since may and all of them have the hung timeout message. Does anyone want to buy a revision A4 that seems to boot fine with the stock angstrom build that comes with the board but won't run debian? I am in Australia but can mail it anywhere.

On Friday, 18 May 2012 21:01:46 UTC+10, Rob Fowler wrote:

I tried this  /3.2-cpufreq-more/  version and have the same crash but now I am looking at the backtrace it is different to yours at the top but still ends up stalling when it 's attempting to schedule io. Mine seems to be locking up scheduling tty io which I assume is eventually going into the USB serial device.

Does anyone know if the wait for completion and schedule encapsulates some sort of interrupt setup and callback? Interrupts not working maybe?

On Friday, 18 May 2012 14:22:01 UTC+10, RobertCNelson wrote:

On Thu, May 17, 2012 at 5:30 PM, Robert Nelson wrote: