Raspberry PI: Interrupts and kernel module hacking
927 views
Skip to first unread message
Andy Gelme
Jun 1, 2014, 5:02:10 AM6/1/14
to Connected Community HackerSpace
hi All,
Yesterday at the CCHS, there was a discussion about Linux and hardware interrupt handling ... for example, using Raspberry Pi GPIO pins.
After digging a bit deeper ...
A simple way to start without having to write a kernel module is to use "user level interrupts", where a user process can register a function to handle a hardware interrupt ...
http://techpubs.sgi.com/library/tpl/cgi-bin/getdoc.cgi?coll=linux&db=bks&srch=&fname=/SGI_Developer/REACTLINUX_PG/sgi_html/ch08.html
A Raspberry Pi specific approach using WiringPi and a small C program ...
http://cs.smith.edu/dftwiki/index.php/Tutorial:_Interrupt-Driven_Event-Counter_on_the_Raspberry_Pi
Or, if you are more comfortable with Python ...
http://raspberrywebserver.com/gpio/using-interrupt-driven-gpio.html
The user-level interrupt approach will work in most cases, where very low-latency is not a requirement, i.e you don't mind up-to 75 microseconds latency.
- - - - - - - - -
If you need to and are willing to get your hands dirty, then writing a Linux kernel module, a hardware interrupt handler and asynchronously notify (signal) your user process may be the way to go. Being able to write code in the kernel module to service the hardware interrupt will reduce latency ... and then you can communicate with your user process in a more leisurely fashion.
Introduction to writing a Linux kernel module ...
http://www.linuxvoice.com/be-a-kernel-hacker
Linux hardware interrupt handling ...
http://www.xml.com/ldd/chapter/book/ch09.html
In the user process, register a signal handler ...
In kernel space, you'll need to send a signal to your user process ...
More details ...
http://www.xml.com/ldd/chapter/book/ch05.html#t4
Yesterday at the CCHS, there was a discussion about Linux and hardware interrupt handling ... for example, using Raspberry Pi GPIO pins.
After digging a bit deeper ...
A simple way to start without having to write a kernel module is to use "user level interrupts", where a user process can register a function to handle a hardware interrupt ...
http://techpubs.sgi.com/library/tpl/cgi-bin/getdoc.cgi?coll=linux&db=bks&srch=&fname=/SGI_Developer/REACTLINUX_PG/sgi_html/ch08.html
A Raspberry Pi specific approach using WiringPi and a small C program ...
http://cs.smith.edu/dftwiki/index.php/Tutorial:_Interrupt-Driven_Event-Counter_on_the_Raspberry_Pi
Or, if you are more comfortable with Python ...
http://raspberrywebserver.com/gpio/using-interrupt-driven-gpio.html
The user-level interrupt approach will work in most cases, where very low-latency is not a requirement, i.e you don't mind up-to 75 microseconds latency.
- - - - - - - - -
If you need to and are willing to get your hands dirty, then writing a Linux kernel module, a hardware interrupt handler and asynchronously notify (signal) your user process may be the way to go. Being able to write code in the kernel module to service the hardware interrupt will reduce latency ... and then you can communicate with your user process in a more leisurely fashion.
Introduction to writing a Linux kernel module ...
http://www.linuxvoice.com/be-a-kernel-hacker
Linux hardware interrupt handling ...
http://www.xml.com/ldd/chapter/book/ch09.html
In the user process, register a signal handler ...
signal(SIGIO, &signal_handler);
fcntl(fd, F_SETOWN, getpid());
oflags = fcntl(fd, F_GETFL);
fcntl(fd, F_SETFL, oflags | FASYNC);void signal_handler(int signal) {
}In kernel space, you'll need to send a signal to your user process ...
struct siginfo info; memset(&info, 0, sizeof(struct siginfo)); info.si_signo = SIGIO; info.si_code = SI_USER;send_sig_info(SIG_TEST, &info,find_task_by_vpid(PROCESS_ID));
More details ...
http://www.xml.com/ldd/chapter/book/ch05.html#t4
-- -O- cheers = /\ /\/ /) `/ = --O -- http://www.geekscape.org -- OOO -- an...@geekscape.org -- http://twitter.com/geekscape --
Angus Gratton
Jun 1, 2014, 8:21:13 PM6/1/14
to connected-commu...@googlegroups.com
Hi Andy,
I didn't witness the discussion in the space so I'm not sure exactly
what was discussed, hopefully these comments aren't too off-base.
On Sun, Jun 01, 2014 at 07:02:07PM +1000, Andy Gelme wrote:
> A simple way to start without having to write a kernel module is to use
> "user level interrupts", where a user process can register a function to
> handle a hardware interrupt ...
From the point of view of an interrupt-oriented coding style, I
totally agree with this approach. However, from the point of view of
interrupts providing timing and latency improvements there are some
traps.
I think the specific User Level Interrupt API functionality mentioned
was only ever a part of SGI's REACT real-time linux. :(
> A Raspberry Pi specific approach using WiringPi and a small C program ...
>
>
> http://cs.smith.edu/dftwiki/index.php/Tutorial:_Interrupt-Driven_Event-Counter_on_the_Raspberry_Pi
Looking in the WiringPi source reveals that registering a WiringPi
interrupt spawns a thread which opens the gpio's sysfs file entry (ie
/sys/class/gpio/gpioXX) and then calls poll() on it, to block and wait
for it to change.
https://git.drogon.net/?p=wiringPi;a=blob;f=wiringPi/wiringPi.c;h=4660a67b47b2c21a9a60a2f8548d6f8b4d2548b4;hb=HEAD#l1259
The difference from SGI's User Level Interrupt is that all user code
(and WiringPi code) runs in the user process. On the kernel side the
gpio interrupt triggers an update from an ISR, but then the Linux
scheduler must schedule the WiringPi process so it can wake up and
respond. Relative to each other, these two operations aren't
guaranteed to fit within any particular time window.
I think the advantage to WiringPi interrupts is to write code that's
more readable written when in an interrupt handler style, and to avoid
burning CPU cycles busy-polling a pin.
> The user-level interrupt approach will work in most cases, where very
> low-latency is not a requirement, i.e you don't mind up-to 75
> microseconds latency.
Absolutely. Though I'd be interested to know how the wiringPi users
measured 75 microseconds of latency. From my experience I'd expect
this to be median latency on a relatively quiet system, not the worst
case latency they claim.
I gave a talk at the LCA2014 Arduino MiniConf "Arduino vs Raspberry Pi
- Decisions, Decisions"
The slides are here: http://projectgus.com/talks/arduino_vs_pi/
... but they're not super intuitive by themselves unfortunately (I
meant to turn them into a blog post but haven't finished it yet):
The DSO captures in the slides show measurements of jitter & worst
case latency in Raspbian. Even with some basic tweaks (like setting
real-time process priority), regular 10 millisecond delays were as
good as it got in those tests.
There are some other more involved tweaks people have used (including
porting the Linux realtime patchset to Raspberry Pi.) Back in January
I think that project was pretty experimental, but things might have
matured since then.
Despite all that, I know some people have had good luck using WiringPi
and interrupts for fairly low latency stuff. For example a Freetronics
customer ported the IRTemp Arduino library that you wrote to WiringPi
(and that clock runs at a fairly high speed):
https://gist.github.com/unprolix/7298736
... and this works, however my guess is it only works most of the
time. Especially if the system becomes loaded up with other processes,
USB or network traffic, etc. then it may miss samples or suddenly stop
working entirely. It works well enough for whatever the author had
in mind, though, so that's good.
> If you need to and are willing to get your hands dirty, then writing a
> Linux kernel module, a hardware interrupt handler and asynchronously
> notify (signal) your user process may be the way to go.
Agreed! It's a shame there's no easier way to get that kind of
performance. :(
Another solution, which I'm sure you discussed in the space as well,
is to combine the complex, non-real-time, system running on the
Raspberry Pi with an attached smaller, simpler, real-time system like
an Arduino. This way each part can deal with the aspect of the problem
that it's best suited for.
This is probably my preferred approach in my own projects, I've found
the other approach quickly develops a feeling of "round peg jammed
into square hole". :)
Cheers,
Angus
I didn't witness the discussion in the space so I'm not sure exactly
what was discussed, hopefully these comments aren't too off-base.
On Sun, Jun 01, 2014 at 07:02:07PM +1000, Andy Gelme wrote:
> A simple way to start without having to write a kernel module is to use
> "user level interrupts", where a user process can register a function to
> handle a hardware interrupt ...
totally agree with this approach. However, from the point of view of
interrupts providing timing and latency improvements there are some
traps.
I think the specific User Level Interrupt API functionality mentioned
was only ever a part of SGI's REACT real-time linux. :(
> A Raspberry Pi specific approach using WiringPi and a small C program ...
>
>
> http://cs.smith.edu/dftwiki/index.php/Tutorial:_Interrupt-Driven_Event-Counter_on_the_Raspberry_Pi
interrupt spawns a thread which opens the gpio's sysfs file entry (ie
/sys/class/gpio/gpioXX) and then calls poll() on it, to block and wait
for it to change.
https://git.drogon.net/?p=wiringPi;a=blob;f=wiringPi/wiringPi.c;h=4660a67b47b2c21a9a60a2f8548d6f8b4d2548b4;hb=HEAD#l1259
The difference from SGI's User Level Interrupt is that all user code
(and WiringPi code) runs in the user process. On the kernel side the
gpio interrupt triggers an update from an ISR, but then the Linux
scheduler must schedule the WiringPi process so it can wake up and
respond. Relative to each other, these two operations aren't
guaranteed to fit within any particular time window.
I think the advantage to WiringPi interrupts is to write code that's
more readable written when in an interrupt handler style, and to avoid
burning CPU cycles busy-polling a pin.
> The user-level interrupt approach will work in most cases, where very
> low-latency is not a requirement, i.e you don't mind up-to 75
> microseconds latency.
measured 75 microseconds of latency. From my experience I'd expect
this to be median latency on a relatively quiet system, not the worst
case latency they claim.
I gave a talk at the LCA2014 Arduino MiniConf "Arduino vs Raspberry Pi
- Decisions, Decisions"
The slides are here: http://projectgus.com/talks/arduino_vs_pi/
... but they're not super intuitive by themselves unfortunately (I
meant to turn them into a blog post but haven't finished it yet):
The DSO captures in the slides show measurements of jitter & worst
case latency in Raspbian. Even with some basic tweaks (like setting
real-time process priority), regular 10 millisecond delays were as
good as it got in those tests.
There are some other more involved tweaks people have used (including
porting the Linux realtime patchset to Raspberry Pi.) Back in January
I think that project was pretty experimental, but things might have
matured since then.
Despite all that, I know some people have had good luck using WiringPi
and interrupts for fairly low latency stuff. For example a Freetronics
customer ported the IRTemp Arduino library that you wrote to WiringPi
(and that clock runs at a fairly high speed):
https://gist.github.com/unprolix/7298736
... and this works, however my guess is it only works most of the
time. Especially if the system becomes loaded up with other processes,
USB or network traffic, etc. then it may miss samples or suddenly stop
working entirely. It works well enough for whatever the author had
in mind, though, so that's good.
> If you need to and are willing to get your hands dirty, then writing a
> Linux kernel module, a hardware interrupt handler and asynchronously
> notify (signal) your user process may be the way to go.
performance. :(
Another solution, which I'm sure you discussed in the space as well,
is to combine the complex, non-real-time, system running on the
Raspberry Pi with an attached smaller, simpler, real-time system like
an Arduino. This way each part can deal with the aspect of the problem
that it's best suited for.
This is probably my preferred approach in my own projects, I've found
the other approach quickly develops a feeling of "round peg jammed
into square hole". :)
Cheers,
Angus
Andy Gelme
Jun 1, 2014, 9:02:59 PM6/1/14
to connected-commu...@googlegroups.com
hi Angus,
On 2014-06-2 10:21 , Angus Gratton wrote:
> hopefully these comments aren't too off-base.
Not at all. The idea was to spark off some informed discussion.
Positive corrections and improvements welcome.
> I think the specific User Level Interrupt API functionality mentioned
> was only ever a part of SGI's REACT real-time linux. :(
Thanks ... I missed that.
I was looking for a reasonable description of Linux user-level
interrupts (there isn't much that I could find).
>> Looking in the WiringPi source reveals that registering a WiringPi
>> interrupt spawns a thread which opens the gpio's sysfs file entry (ie
>> /sys/class/gpio/gpioXX) and then calls poll() on it, to block and wait
>> for it to change.
Saturday's discussion did touch upon using the file system as a
potential means of receiving an "interrupt" at the user level. I wasn't
keen on the idea ... and would rather suggest to just write a proper
kernel module and a hardware interrupt handler, but that isn't advice
that is easy to run with (without decent documentation and examples).
For someone just getting into this area, WiringPi does make it easier to
start. As you point out, at the cost of performance ...
> ... WiringPi code) runs in the user process. On the kernel side the
> I gave a talk at the LCA2014 Arduino MiniConf "Arduino vs Raspberry Pi - Decisions, Decisions"
> The slides are here: http://projectgus.com/talks/arduino_vs_pi
It's a shame the discussion happened after you left :(
> The DSO captures in the slides show measurements of jitter & worst
> case latency in Raspbian. Even with some basic tweaks (like setting
> real-time process priority), regular 10 millisecond delays were as
> good as it got in those tests
For many simple applications, 10 ms is an acceptable delay.
However, it does consume a relatively large percentage of the Raspberry
Pi CPU ... compared to a dedicated microcontroller which has a response
time measured in microseconds and very low CPU overhead.
> Another solution, which I'm sure you discussed in the space as well,
> is to combine the complex, non-real-time, system running on the
> Raspberry Pi with an attached smaller, simpler, real-time system like
> an Arduino. This way each part can deal with the aspect of the problem
> that it's best suited for. This is probably my preferred approach in
> my own projects, I've found the other approach quickly develops a
> feeling of "round peg jammed into square hole". :)
We didn't discuss the hybrid approach, due to the nature of the original
question ... which was along the lines of how can you do this on a Linux
system (perhaps using the /proc or /sys file-systems).
But, I'm with you 100% on using a hybrid Linux / dedicated
microcontroller approach. You get the best of both worlds ... at a
low-cost. You can pick up a Trinket for US$5 to prototype and hook it
up to the Raspberry Pi serial UART (on the GPIO pins). When making more
devices, then an ATTiny costs around 80 cents.
As very briefly implied at the end of your slides ... you can do some
really interesting things with a BeagleBoard Black with the built-in
RTUs. Which is another form of hybrid solution (not as well travelled
as the Raspberry Pi / Arduino hybrid).
On 2014-06-2 10:21 , Angus Gratton wrote:
> hopefully these comments aren't too off-base.
Positive corrections and improvements welcome.
> I think the specific User Level Interrupt API functionality mentioned
> was only ever a part of SGI's REACT real-time linux. :(
I was looking for a reasonable description of Linux user-level
interrupts (there isn't much that I could find).
>> Looking in the WiringPi source reveals that registering a WiringPi
>> interrupt spawns a thread which opens the gpio's sysfs file entry (ie
>> /sys/class/gpio/gpioXX) and then calls poll() on it, to block and wait
>> for it to change.
potential means of receiving an "interrupt" at the user level. I wasn't
keen on the idea ... and would rather suggest to just write a proper
kernel module and a hardware interrupt handler, but that isn't advice
that is easy to run with (without decent documentation and examples).
For someone just getting into this area, WiringPi does make it easier to
start. As you point out, at the cost of performance ...
> ... WiringPi code) runs in the user process. On the kernel side the
> gpio interrupt triggers an update from an ISR, but then the Linux
> scheduler must schedule the WiringPi process so it can wake up and
> respond. Relative to each other, these two operations aren't
> guaranteed to fit within any particular time window.
... and unpredictable maximum latency.
> scheduler must schedule the WiringPi process so it can wake up and
> respond. Relative to each other, these two operations aren't
> guaranteed to fit within any particular time window.
> I gave a talk at the LCA2014 Arduino MiniConf "Arduino vs Raspberry Pi - Decisions, Decisions"
> The slides are here: http://projectgus.com/talks/arduino_vs_pi
> The DSO captures in the slides show measurements of jitter & worst
> case latency in Raspbian. Even with some basic tweaks (like setting
> real-time process priority), regular 10 millisecond delays were as
> good as it got in those tests
However, it does consume a relatively large percentage of the Raspberry
Pi CPU ... compared to a dedicated microcontroller which has a response
time measured in microseconds and very low CPU overhead.
> Another solution, which I'm sure you discussed in the space as well,
> is to combine the complex, non-real-time, system running on the
> Raspberry Pi with an attached smaller, simpler, real-time system like
> an Arduino. This way each part can deal with the aspect of the problem
> that it's best suited for. This is probably my preferred approach in
> my own projects, I've found the other approach quickly develops a
> feeling of "round peg jammed into square hole". :)
question ... which was along the lines of how can you do this on a Linux
system (perhaps using the /proc or /sys file-systems).
But, I'm with you 100% on using a hybrid Linux / dedicated
microcontroller approach. You get the best of both worlds ... at a
low-cost. You can pick up a Trinket for US$5 to prototype and hook it
up to the Raspberry Pi serial UART (on the GPIO pins). When making more
devices, then an ATTiny costs around 80 cents.
As very briefly implied at the end of your slides ... you can do some
really interesting things with a BeagleBoard Black with the built-in
RTUs. Which is another form of hybrid solution (not as well travelled
as the Raspberry Pi / Arduino hybrid).
Clifford Heath
Jun 1, 2014, 9:22:51 PM6/1/14
to connected-commu...@googlegroups.com
Regarding measuring latency, I don’t know what hardware this was on,
but it’s trivial on the x86; use the TSC register. This is a 64-bit register
that counts at the clock speed, so you get sub-ms level timing resolution.
You could record the TSC register on entry to the interrupt, and again on
entry to the user-level interrupt, and subtract them.
Perhaps the SGI hardware had a register like this too? Though perhaps
not the Pi?
Clifford Heath.
> --
> You received this message because you are subscribed to the Google Groups "Connected Community HackerSpace" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to connected-community-h...@googlegroups.com.
> To post to this group, send an email to connected-commu...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/connected-community-hackerspace/538BCD40.7010901%40geekscape.org.
> For more options, visit https://groups.google.com/d/optout.
but it’s trivial on the x86; use the TSC register. This is a 64-bit register
that counts at the clock speed, so you get sub-ms level timing resolution.
You could record the TSC register on entry to the interrupt, and again on
entry to the user-level interrupt, and subtract them.
Perhaps the SGI hardware had a register like this too? Though perhaps
not the Pi?
Clifford Heath.
> You received this message because you are subscribed to the Google Groups "Connected Community HackerSpace" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to connected-community-h...@googlegroups.com.
> To post to this group, send an email to connected-commu...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/connected-community-hackerspace/538BCD40.7010901%40geekscape.org.
> For more options, visit https://groups.google.com/d/optout.
Andy Gelme
Jun 1, 2014, 10:25:40 PM6/1/14
to connected-commu...@googlegroups.com
hi All,
On 2014-06-2 11:22 , Clifford Heath wrote:
> Regarding measuring latency, I
don’t know what hardware this was on,
> but it’s trivial on the x86; use the TSC register ...
Lots of people using ARM, AVR and other non-x86 processors. On larger microcontrollers, there may be a spare timer/counter register that you can use (similar to your TSC suggestion) ... but, not-so-often on the smaller microcontrollers.
A easy technique which works on all hardware is to simply pulse or toggle a spare I/O pin ... at an appropriate point, e.g when your hardware interrupt handler gets invoked (turn pin on) and when your (user-level) application handler gets invoked (turn pin off). Then, watch that output pin on an oscilloscope or logic analyser.
Often, using a serial console for timing information / diagnosis isn't available / accessible ... or outputing serial data will perturb the timing that you are trying to carefully measure.
Lots of people using ARM, AVR and other non-x86 processors. On larger microcontrollers, there may be a spare timer/counter register that you can use (similar to your TSC suggestion) ... but, not-so-often on the smaller microcontrollers.
A easy technique which works on all hardware is to simply pulse or toggle a spare I/O pin ... at an appropriate point, e.g when your hardware interrupt handler gets invoked (turn pin on) and when your (user-level) application handler gets invoked (turn pin off). Then, watch that output pin on an oscilloscope or logic analyser.
Often, using a serial console for timing information / diagnosis isn't available / accessible ... or outputing serial data will perturb the timing that you are trying to carefully measure.
nog3
Jun 1, 2014, 11:13:18 PM6/1/14
to connected-commu...@googlegroups.com
This discussion makes me wonder if the Intel Galileo has the tsc register.
--
You received this message because you are subscribed to the Google Groups "Connected Community HackerSpace" group.
To unsubscribe from this group and stop receiving emails from it, send an email to connected-community-h...@googlegroups.com.
To post to this group, send email to connected-commu...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/connected-community-hackerspace/538BE09D.50507%40geekscape.org.
Chris Fryer
Jun 2, 2014, 12:17:01 AM6/2/14
to connected-commu...@googlegroups.com
Funny you say that as I have been playing around with an Intel Galileo I recently bought for myself for my birthday. I didn't really know what to do with it, but over the last few days I have Installed a larger version of linux onto an SD card because I wasn't satisfied with the cutdown version that comes preinstalled. Today I succeeded in enabling the Wi-Fi capability and am able to use telnet remotely.
Almost all of what you guys have said here goes straight over my head as I have only been using Linux for less than a week. The good thing about the Galileo is you can upload Arduino sketches through a modified Arduino IDE. It is also hardware compatible with most Arduino shields. The Galileo is really a best of both worlds thing, but thereafter a couple of drawbacks as it really emulates an Arduino meaning that sketches run considerably slower.
Don't know what it said helps. I still don't know what to do with it, any ideas. :P
pete...@gmail.com
Jun 2, 2014, 9:40:43 PM6/2/14
to connected-commu...@googlegroups.com
I had never heard of it till now. There is also a ninja block which is a arduino shield for pi . Slightly cheaper option. $54 + pi cost Designed in NSW
Reply all
Reply to author
Forward
0 new messages