Xenomai - plantage du kernel avec mon module

66 views
Skip to first unread message

AleaJactaEst

unread,
Oct 15, 2012, 4:29:14 PM10/15/12
to concours-li...@googlegroups.com
Bonjour,

Je cherche une méthode pour identifier un plantage du kernel suite à l'exécution de mon module Xenomai.
En résumé, mon module [xeno_s3c2440_adc] plante tout au bout de quelques (longue) minutes.

Et je n'arrive pas à identifier l'origine du problème.

---------------------------
[  810.390000] Unable to handle kernel NULL pointer dereference at virtual address 00000030
[  810.390000] fcse pid: 0, 0x00000000
[  810.390000] pgd = c3acc000
[  810.390000] [00000030] *pgd=33aec831, *pte=00000000, *ppte=00000000
[  810.390000] Internal error: Oops: 17 [#1] PREEMPT
[  810.390000] Modules linked in: xeno_s3c2440_adc
[  810.390000] CPU: 0    Tainted: G        W     (3.2.21 #5)
[  810.390000] PC is at check_preempt_tick.clone.133+0x54/0xe8
[  810.390000] LR is at check_preempt_tick.clone.133+0x1c/0xe8
[  810.390000] pc : [<c0017bc4>]    lr : [<c0017b8c>]    psr: 60000013
[  810.390000] sp : c3b17ee0  ip : c38c3730  fp : c3b17eec
[  810.390000] r10: 0000001e  r9 : c046c37c  r8 : 00000000
[  810.390000] r7 : 00000000  r6 : 00000001  r5 : c0469300  r4 : c38c3730
[  810.390000] r3 : 00000000  r2 : 00000030  r1 : 00000000  r0 : 7fffffff
[  810.390000] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[  810.390000] Control: c000717f  Table: 33acc000  DAC: 00000015
[  810.390000] Process oscilloscope_al (pid: 782, stack limit = 0xc3b16270)
[  810.390000] Stack: (0xc3b17ee0 to 0xc3b18000)
[  810.390000] 7ee0: c0469300 c38c3700 c3b17efc c00196ac c38c3700 c3b16000 c04c64f8 c00293dc
[  810.390000] 7f00: c0467270 c0467270 00000000 c046df14 00000000 c0014dec 00800000 c004f348
[  810.390000] 7f20: 60000013 00000000 c04c44a8 c046c37c c0467270 00000000 c046e3dc c04c6540
[  810.390000] 7f40: 00000000 c3b16000 004a3318 c004f510 00020000 c046c37c 0000001e c0052198
[  810.390000] 7f60: c0474a74 c004ec6c 00000055 c0009bec c04c6540 00000001 c046df14 c00568d4
[  810.390000] 7f80: 0000001e 0000001f c3b17fb0 60000010 f6000000 004a3318 004a3330 004a3330
[  810.390000] 7fa0: 01dc0a34 c0008378 00c35ecc c0009094 005b6f00 005b6f00 00000000 00000000
[  810.390000] 7fc0: 005b6f00 004a3348 004a3280 004a3318 004a3330 004a3330 01dc0a34 004a3318
[  810.390000] 7fe0: 00000000 01dc0938 00c366fc 00c35ecc 60000010 ffffffff 00000000 00000000
[  810.390000] [<c0017bc4>] (check_preempt_tick.clone.133+0x54/0xe8) from [<c00196ac>] (scheduler_tick+0x40/0x74)
[  810.390000] [<c00196ac>] (scheduler_tick+0x40/0x74) from [<c00293dc>] (update_process_times+0x50/0x5c)
[  810.390000] [<c00293dc>] (update_process_times+0x50/0x5c) from [<c0014dec>] (s3c2410_timer_interrupt+0x40/0x4c)
[  810.390000] [<c0014dec>] (s3c2410_timer_interrupt+0x40/0x4c) from [<c004f348>] (handle_irq_event_percpu+0x50/0x1c8)
[  810.390000] [<c004f348>] (handle_irq_event_percpu+0x50/0x1c8) from [<c004f510>] (handle_irq_event+0x50/0x78)
[  810.390000] [<c004f510>] (handle_irq_event+0x50/0x78) from [<c0052198>] (handle_level_irq+0x68/0xd4)
[  810.390000] [<c0052198>] (handle_level_irq+0x68/0xd4) from [<c004ec6c>] (generic_handle_irq+0x20/0x34)
[  810.390000] [<c004ec6c>] (generic_handle_irq+0x20/0x34) from [<c0009bec>] (handle_IRQ+0x30/0x84)
[  810.390000] [<c0009bec>] (handle_IRQ+0x30/0x84) from [<c00568d4>] (__ipipe_do_sync_stage+0x148/0x1dc)
[  810.390000] [<c00568d4>] (__ipipe_do_sync_stage+0x148/0x1dc) from [<c0008378>] (__ipipe_grab_irq+0x28/0x5c)
[  810.390000] [<c0008378>] (__ipipe_grab_irq+0x28/0x5c) from [<c0009094>] (__irq_usr+0x34/0xc8)
[  810.390000] Exception stack(0xc3b17fb0 to 0xc3b17ff8)
[  810.390000] 7fa0:                                     005b6f00 005b6f00 00000000 00000000
[  810.390000] 7fc0: 005b6f00 004a3348 004a3280 004a3318 004a3330 004a3330 01dc0a34 004a3318
[  810.390000] 7fe0: 00000000 01dc0938 00c366fc 00c35ecc 60000010 ffffffff
[  810.390000] Code: e3530000 12432008 01a02003 e2822030 (e8920006)
[  810.390000] ---[ end trace d55ad39bfa4c8594 ]---
[  810.390000] Kernel panic - not syncing: Fatal exception in interrupt
[  810.390000] [<c000da5c>] (unwind_backtrace+0x0/0xf0) from [<c0349724>] (panic+0x60/0x190)
[  810.390000] [<c0349724>] (panic+0x60/0x190) from [<c000c5c0>] (die+0x114/0x12c)
[  810.390000] [<c000c5c0>] (die+0x114/0x12c) from [<c000f8b8>] (__do_kernel_fault+0x64/0x84)
[  810.390000] [<c000f8b8>] (__do_kernel_fault+0x64/0x84) from [<c000fa20>] (do_page_fault+0x148/0x24c)
[  810.390000] [<c000fa20>] (do_page_fault+0x148/0x24c) from [<c00083ec>] (do_DataAbort+0x40/0xf4)
[  810.390000] [<c00083ec>] (do_DataAbort+0x40/0xf4) from [<c0008e6c>] (__dabt_svc+0x2c/0x40)
[  810.390000] Exception stack(0xc3b17e98 to 0xc3b17ee0)
[  810.390000] 7e80:                                                       7fffffff 00000000
[  810.390000] 7ea0: 00000030 00000000 c38c3730 c0469300 00000001 00000000 00000000 c046c37c
[  810.390000] 7ec0: 0000001e c3b17eec c38c3730 c3b17ee0 c0017b8c c0017bc4 60000013 ffffffff
[  810.390000] [<c0008e6c>] (__dabt_svc+0x2c/0x40) from [<c0017bc4>] (check_preempt_tick.clone.133+0x54/0xe8)
[  810.390000] [<c0017bc4>] (check_preempt_tick.clone.133+0x54/0xe8) from [<c00196ac>] (scheduler_tick+0x40/0x74)
[  810.390000] [<c00196ac>] (scheduler_tick+0x40/0x74) from [<c00293dc>] (update_process_times+0x50/0x5c)
[  810.390000] [<c00293dc>] (update_process_times+0x50/0x5c) from [<c0014dec>] (s3c2410_timer_interrupt+0x40/0x4c)
[  810.390000] [<c0014dec>] (s3c2410_timer_interrupt+0x40/0x4c) from [<c004f348>] (handle_irq_event_percpu+0x50/0x1c8)
[  810.390000] [<c004f348>] (handle_irq_event_percpu+0x50/0x1c8) from [<c004f510>] (handle_irq_event+0x50/0x78)
[  810.390000] [<c004f510>] (handle_irq_event+0x50/0x78) from [<c0052198>] (handle_level_irq+0x68/0xd4)
[  810.390000] [<c0052198>] (handle_level_irq+0x68/0xd4) from [<c004ec6c>] (generic_handle_irq+0x20/0x34)
[  810.390000] [<c004ec6c>] (generic_handle_irq+0x20/0x34) from [<c0009bec>] (handle_IRQ+0x30/0x84)
[  810.390000] [<c0009bec>] (handle_IRQ+0x30/0x84) from [<c00568d4>] (__ipipe_do_sync_stage+0x148/0x1dc)
[  810.390000] [<c00568d4>] (__ipipe_do_sync_stage+0x148/0x1dc) from [<c0008378>] (__ipipe_grab_irq+0x28/0x5c)
[  810.390000] [<c0008378>] (__ipipe_grab_irq+0x28/0x5c) from [<c0009094>] (__irq_usr+0x34/0xc8)
[  810.390000] Exception stack(0xc3b17fb0 to 0xc3b17ff8)
[  810.390000] 7fa0:                                     005b6f00 005b6f00 00000000 00000000
[  810.390000] 7fc0: 005b6f00 004a3348 004a3280 004a3318 004a3330 004a3330 01dc0a34 004a3318
[  810.390000] 7fe0: 00000000 01dc0938 00c366fc 00c35ecc 60000010 ffffffff
---------------------------------------

@+
AleaJactaEst

AleaJactaEst

unread,
Oct 15, 2012, 4:31:53 PM10/15/12
to concours-li...@googlegroups.com
Et voilà un autre plantage


[ 3190.025000] Unable to handle kernel NULL pointer dereference at virtual address 00000008
[ 3190.030000] fcse pid: 0, 0x00000000
[ 3190.030000] pgd = c0004000
[ 3190.035000] [00000008] *pgd=00000000
[ 3190.040000] Internal error: Oops: 17 [#1] PREEMPT
[ 3190.040000] Modules linked in: xeno_s3c2440_adc
[ 3190.040000] CPU: 0    Tainted: G        W     (3.2.21 #5)
[ 3190.040000] PC is at rb_next+0x0/0x68
[ 3190.040000] LR is at pick_next_entity+0x8c/0xbc
[ 3190.040000] pc : [<c01ed898>]    lr : [<c0016528>]    psr: 60000013
[ 3190.040000] sp : c386def8  ip : 00000000  fp : c386df0c
[ 3190.040000] r10: 00000000  r9 : c384e720  r8 : 00000000
[ 3190.040000] r7 : c0469300  r6 : 00000000  r5 : 00000000  r4 : c0469338
[ 3190.040000] r3 : 00000001  r2 : 00000000  r1 : c384e5c0  r0 : 00000008
[ 3190.040000] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment kernel
[ 3190.040000] Control: c000717f  Table: 33af8000  DAC: 00000017
[ 3190.040000] Process sync_supers (pid: 137, stack limit = 0xc386c270)
[ 3190.040000] Stack: (0xc386def8 to 0xc386e000)
[ 3190.040000] dee0:                                                       c0469300 c384e5c0
[ 3190.040000] df00: c0469338 c0469300 c386df24 c001657c c034d708 c384e5c0 c386c000 c0469300
[ 3190.040000] df20: c386dfb4 c0349f68 00000000 c046df14 c386c000 f6000000 c386dfa4 33af8000
[ 3190.040000] df40: c386c000 00000000 c386df5c c034a838 c00378e0 00000000 00000000 c034bf3c
[ 3190.040000] df60: c046df14 c00378e0 40000013 c0008f4c 00000000 00000001 c386dfb8 c386dfd4
[ 3190.040000] df80: c386c000 00000001 00000000 c00b30d4 00000000 c386c000 00000001 00000000
[ 3190.040000] dfa0: c00b30d4 00000000 00000000 00000000 00000000 c00b3104 00000000 c3827f70
[ 3190.040000] dfc0: c386dfd4 c0037d68 c0009ca4 00000000 00000000 00000000 c386dfd8 c386dfd8
[ 3190.040000] dfe0: 00000000 c3827f70 c0037ce0 c0009ca4 00000013 c0009ca4 00000000 00000000
[ 3190.040000] [<c01ed898>] (rb_next+0x0/0x68) from [<c0016528>] (pick_next_entity+0x8c/0xbc)
[ 3190.040000] [<c0016528>] (pick_next_entity+0x8c/0xbc) from [<c001657c>] (pick_next_task_fair+0x24/0x8c)
[ 3190.040000] [<c001657c>] (pick_next_task_fair+0x24/0x8c) from [<c0349f68>] (__schedule+0x4f8/0x638)
[ 3190.040000] [<c0349f68>] (__schedule+0x4f8/0x638) from [<c00b3104>] (bdi_sync_supers+0x30/0x48)
[ 3190.040000] [<c00b3104>] (bdi_sync_supers+0x30/0x48) from [<c0037d68>] (kthread+0x88/0x90)
[ 3190.040000] [<c0037d68>] (kthread+0x88/0x90) from [<c0009ca4>] (kernel_thread_exit+0x0/0x8)
[ 3190.040000] Code: e5903004 e3530000 1afffffb e12fff1e (e5901000)
[ 3190.045000] ---[ end trace ab38f64fdfff4237 ]---
[ 3190.045000] note: sync_supers[137] exited with preempt_count 2

@+
AleaJactaEst

Romain Naour

unread,
Oct 15, 2012, 5:11:56 PM10/15/12
to concours-li...@googlegroups.com
Bonjour,

As tu essayé d'augmenter CONFIG_XENO_OPT_SYS_STACKPOOLSZ et/ou CONFIG_XENO_OPT_SYS_HEAPSZ ?

A+
Romain

AleaJactaEst

unread,
Oct 15, 2012, 5:35:35 PM10/15/12
to concours-li...@googlegroups.com
Merci pour l'information.

Je viens de faire le test.
CONFIG_XENO_OPT_SYS_STACKPOOLSZ=512
CONFIG_XENO_OPT_SYS_HEAPSZ=512

Mais j'ai toujours les mêmes problèmes.

Je regarde actuellement mon code, je pense qu'il s'agit de spin_lock.
Je ne suis pas sur que cela fonctionne correctement avec Xenomai.
Je vais le transformer en mutex [rt_mutex_acquire]

@+
AleaJactaEst

AleaJactaEst

unread,
Oct 16, 2012, 4:04:09 PM10/16/12
to concours-li...@googlegroups.com
Bonjour,

J'ai toujours mon problème.

J'ai eu aussi le message suivant (qui ne plante pas la machine)
[   16.185000] ------------[ cut here ]------------
[   16.185000] WARNING: at kernel/sched.c:4286 auto_scan_adc+0xc8/0x37c [xeno_s3c2440_adc]()
[   16.185000] Modules linked in: xeno_s3c2440_adc
[   16.185000] [<c000da5c>] (unwind_backtrace+0x0/0xf0) from [<c001d198>] (warn_slowpath_common+0x48/0x60)
[   16.185000] [<c001d198>] (warn_slowpath_common+0x48/0x60) from [<c001d1cc>] (warn_slowpath_null+0x1c/0x24)
[   16.185000] [<c001d1cc>] (warn_slowpath_null+0x1c/0x24) from [<bf00291c>] (auto_scan_adc+0xc8/0x37c [xeno_s3c2440_adc])
[   16.185000] [<bf00291c>] (auto_scan_adc+0xc8/0x37c [xeno_s3c2440_adc]) from [<c0073b98>] (xnarch_thread_trampoline+0x20/0x2c)
[   16.185000] ---[ end trace 0f092f4b0ed9f256 ]---

J'ai eu aussi un autre message (qui à bloquer la mini2440)
# [ 2650.770000] Xenomai: suspending kernel thread bf008eb0 ('s3c2440-adc') at 0xc01ed2d8 after exception #0x0

Mais je ne sais toujours pas comment déboguer tous ces messages.
Si quelqu'un à une méthode, je suis preneur.

PS. :
   auto_scan_adc est une fonction de mon module xeno_s3c2440_adc.

@+
AleaJactaEst

Laurent Navet

unread,
Oct 16, 2012, 4:24:30 PM10/16/12
to concours-li...@googlegroups.com
Bonsoir,

Pas évident sans voir ton code, mais peut-être que ceci peut t'aider :

http://www.barrgroup.com/Embedded-Systems/How-To/C-Volatile-Keyword

2012/10/16, AleaJactaEst <jean.so...@laposte.net>:
--
« On ne résout pas un problème avec les modes de pensée qui l’ont engendré. »
« You cannot solve current problems with current thinking. Current
problems are the result of current thinking »

AleaJactaEst

unread,
Oct 16, 2012, 4:56:25 PM10/16/12
to concours-li...@googlegroups.com
Bonjour,

Concernant le message suivant, il est émis pendant l'exécution de la fonction spin_lock.

[   16.185000] ------------[ cut here ]------------
[   16.185000] WARNING: at kernel/sched.c:4286 auto_scan_adc+0xc8/0x37c [xeno_s3c2440_adc]()
[   16.185000] Modules linked in: xeno_s3c2440_adc
[   16.185000] [<c000da5c>] (unwind_backtrace+0x0/0xf0) from [<c001d198>] (warn_slowpath_common+0x48/0x60)
[   16.185000] [<c001d198>] (warn_slowpath_common+0x48/0x60) from [<c001d1cc>] (warn_slowpath_null+0x1c/0x24)
[   16.185000] [<c001d1cc>] (warn_slowpath_null+0x1c/0x24) from [<bf00291c>] (auto_scan_adc+0xc8/0x37c [xeno_s3c2440_adc])
[   16.185000] [<bf00291c>] (auto_scan_adc+0xc8/0x37c [xeno_s3c2440_adc]) from [<c0073b98>] (xnarch_thread_trampoline+0x20/0x2c)
[   16.185000] ---[ end trace 0f092f4b0ed9f256 ]---
 
@+
AleaJactaEst

AleaJactaEst

unread,
Oct 17, 2012, 5:42:01 PM10/17/12
to concours-li...@googlegroups.com
Bonjour,

je vous envoie le module en question.
(avec beaucoup d'option inutile.

On retrouve dans le code ma tentative d'utilisation du mutex xenomai (désactivé - voir les options "#define" pour le réactiver)
Mais cela ne marche pas dans une utilisation via un device (rt_mutex_acquire renvoie -EPERM)

@+
AleaJactaEst
xenomai_s3c2440_adc_v3.c

AleaJactaEst

unread,
Oct 19, 2012, 3:40:29 PM10/19/12
to concours-li...@googlegroups.com
Bonjour,

J'ai remplacé la fonction ioremap par ioremap_nocache.
J'ai eu un beau succès hier (plusieurs heures de fonctionnement sans plantage)
Depuis, retour à la case de départ. (plantage au bout de quelques minutes)

Pour ceux que cela intéresse, il y a un moyen de retrouver la localisation du plantage en utilisant le programme arm-linux-addr2line.
(compilé dans la chaîne buildroot)

Il faut récupérer la localisation de la fonction qui a planté (récupérer le registre "pc" afficher ans le message oops)

exemple de commande :
  output/host/usr/bin/arm-linux-addr2line -f -e ./output/build/linux-3.2.21/vmlinux c027fb38

@+
AleaJactaest

XIE Enming

unread,
Nov 5, 2012, 6:43:39 PM11/5/12
to concours-li...@googlegroups.com
xeno_s3c2440_adc   c'est quoi ?
Tu l'ecris ou tu le recupere d'autre part?
merci!

AleaJactaEst

unread,
Nov 6, 2012, 5:00:13 PM11/6/12
to concours-li...@googlegroups.com
Bonjour,

xeno_s3c2440_adc est un module que j'ai crée.
Tu retrouve les sources un peu plus haut.

(Par contre, il comporte un bug, en suivant le conseil que j'ai mis après, cela devrait fonctionner sans problème - du moins je n'ai pas constaté d'erreur)

@+
AleaJactaEst

XIE Enming

unread,
Nov 6, 2012, 6:50:55 PM11/6/12
to concours-li...@googlegroups.com
Merci , je vais voir .
Reply all
Reply to author
Forward
0 new messages