More embedded GCC: ARM

[Michael Ashton]

I work with embedded systems. My projects can get complex, and tend to
put a strain on Make and friends, so I've been using SCons. I've found
SCons to be capable, but difficult to work with, so I am shopping for
alternatives.

Recently I found fbuild, and I really like the concepts behind it. It
might be much simpler to use for my work than SCons has been. So I
would like to try to use it to build the firmware for an ARM7TDMI-
based embedded system I'm working on.

My compiler is the Lite version of CodeSourcery G++ for ARM EABI (see
http://www.codesourcery.com/sgpp/lite/arm). This is a (free)
distribution of GCC/G++. The binaries are the regular GCC binaries
prefixed with arm-none-eabi-*.

In this environment, objects are built from C and C++ files as usual,
then linked into an ELF file (suffix .elf). This ELF file can be used
with GDB to download a code image to the target system. Often, the ELF
file is converted to a format like Intel Hex, for use with programmer
systems; this can be done with objcopy. (All of this should be quite
similar to the existing AVR toolchain environment shipping with fbuild
now -- this is one of the things that particularly excited me about
fbuild.)

So, a few questions:

* What's the appropriate way to handle a raft of alternatively-named
GCC and Binutils programs like this? I'm thinking of adding a "bin-
prefix" option to the main GCC builder -- would this be appropriate?
This could, I think, also serve the AVR builder, and there are other
distributions of GCC which have the same property, like MSP430 GCC. I
could submit a patch / fork if so.

* In the ARM7TDMI environment, it is usually necessary to compile
different parts of the code with different options. I need to be able
to compile a certain set of objects with one set of options, another
set with different options, and finally link these objects together
into the final ELF. How do I move these lists of objects around? The
right way to handle this in fbuild isn't clear to me.

* Less important to me but still interesting: Obviously the usual
configuration tests won't work, but some of them might be useful -- it
could be helpful to verify that the compiler at least runs. Where
would be the right place to implement those tests? Would this work
somehow with the cross_compiler option?

Thanks for all the work on this -- fbuild looks great and I'm looking
forward to putting it through its paces.

--Michael

[Erick Tryzelaar]

Hi Michael!

On Thu, Jan 27, 2011 at 5:37 PM, Michael Ashton <tvl...@gmail.com> wrote:
>
> My compiler is the Lite version of CodeSourcery G++ for ARM EABI (see
> http://www.codesourcery.com/sgpp/lite/arm). This is a (free)
> distribution of GCC/G++. The binaries are the regular GCC binaries
> prefixed with arm-none-eabi-*.
>
> In this environment, objects are built from C and C++ files as usual,
> then linked into an ELF file (suffix .elf). This ELF file can be used
> with GDB to download a code image to the target system. Often, the ELF
> file is converted to a format like Intel Hex, for use with programmer
> systems; this can be done with objcopy. (All of this should be quite
> similar to the existing AVR toolchain environment shipping with fbuild
> now -- this is one of the things that particularly excited me about
> fbuild.)

Great! Unfortunately I'm not too familiar with embedded programming,
so I had some help with setting up the AVR toolchain, and I'm not sure
if it's in a finished state. Marek, are you still around? If so, are
you still using fbuild?

Either way, I'm happy to help out. First off, can you either point me
to a copy of your scons files, or give me some example commands you
have to run to get your code running?

> So, a few questions:
>
> * What's the appropriate way to handle a raft of alternatively-named
> GCC and Binutils programs like this? I'm thinking of adding a "bin-
> prefix" option to the main GCC builder -- would this be appropriate?
> This could, I think, also serve the AVR builder, and there are other
> distributions of GCC which have the same property, like MSP430 GCC. I
> could submit a patch / fork if so.

I'm not sure if that's needed. Assuming your version of gcc acts just
like a vanilla version of gcc (ie, no extra flags), I think the best
solution would be to use the default_exes keyword argument. Since you
know you need to use gcc, I'm guessing you can start out with:

import fbuild.builders.c.gcc

def build(ctx):
builder = fbuild.builders.c.gcc.static(ctx,
default_exes=['rm-none-eabi-gcc'],
cross_compiler=True)
...

> * In the ARM7TDMI environment, it is usually necessary to compile
> different parts of the code with different options. I need to be able
> to compile a certain set of objects with one set of options, another
> set with different options, and finally link these objects together
> into the final ELF. How do I move these lists of objects around? The
> right way to handle this in fbuild isn't clear to me.

Yeah, the documentation needs more work, I'll make sure to add a
section describing a scenario like this. Assuming you've got a
builder, you should be able to do things like:

objs1 = builder.build_objs(['a.c', 'b.c'], optimize=True,
macros=['FOO', 'BAR'])
objs2 = builder.build_objs(['c.c', 'd.c'], debug=True, macros=['BAZ'])
exe = builder.link_exe('exe', objs1 + objs2)

Or you can configure two builders:

builder1 = fbuild.builders.c.gcc.static(ctx, ..., optimize=True,
macros=['FOO', 'BAR'])
objs1 = builder2.build_objs(['a.c', 'b.c'])

builder2 = fbuild.builders.c.gcc.static(ctx, ..., debug=True,
macros=['BAZ'])
objs2 = builder2.build_objs(['c.c', 'd.c'])

builder3 = fbuild.builders.c.gcc.static(ctx, ...)
exe = builder3.link_exe('exe', objs1 + objs2)

> * Less important to me but still interesting: Obviously the usual
> configuration tests won't work, but some of them might be useful -- it
> could be helpful to verify that the compiler at least runs. Where
> would be the right place to implement those tests? Would this work
> somehow with the cross_compiler option?

I wasn't sure how to test cross compiled executables in the general
case, so fbuild only tests if the compiler can build things, but it
doesn't actually make sure those things can execute. If you know how
to run your binary though, you can do:

def build(ctx):
builder = ...
exe = builder.build_exe(...)

# I'll pretend there's some simulator out there that can run your binary
ctx.looger.check('checking if we can run {}'.format(exe))
try:
ctx.execute(['/usr/bin/simulator', exe])
except fbuild.ExecutionError:
# give up because we failed
ctx.logger.failed()
return
else:
ctx.logger.passed()

> Thanks for all the work on this -- fbuild looks great and I'm looking
> forward to putting it through its paces.

Thanks for trying it out! Please let me know if you have more questions.

[Marek Kubica]

On Thu, 27 Jan 2011 18:13:22 -0800
Erick Tryzelaar <erick.t...@gmail.com> wrote:

> Great! Unfortunately I'm not too familiar with embedded programming,
> so I had some help with setting up the AVR toolchain, and I'm not sure
> if it's in a finished state. Marek, are you still around? If so, are
> you still using fbuild?

Yes I'm around, but as that project is done, I haven't done anything
with AVR since. Which is a shame, but I got quite busy with other stuff.

regards,
Marek

[Michael Ashton]

On Thu, Jan 27, 2011 at 6:13 PM, Erick Tryzelaar <erick.t...@gmail.com> wrote:

Hi Michael!

On Thu, Jan 27, 2011 at 5:37 PM, Michael Ashton <tvl...@gmail.com> wrote:
>
> My compiler is the Lite version of CodeSourcery G++ for ARM EABI (see
> http://www.codesourcery.com/sgpp/lite/arm). This is a (free)
> distribution of GCC/G++. The binaries are the regular GCC binaries
> prefixed with arm-none-eabi-*.
>
> In this environment, objects are built from C and C++ files as usual,
> then linked into an ELF file (suffix .elf). This ELF file can be used
> with GDB to download a code image to the target system. Often, the ELF
> file is converted to a format like Intel Hex, for use with programmer
> systems; this can be done with objcopy. (All of this should be quite
> similar to the existing AVR toolchain environment shipping with fbuild
> now -- this is one of the things that particularly excited me about
> fbuild.)

Great! Unfortunately I'm not too familiar with embedded programming,
so I had some help with setting up the AVR toolchain, and I'm not sure
if it's in a finished state. Marek, are you still around? If so, are
you still using fbuild?

Either way, I'm happy to help out. First off, can you either point me
to a copy of your scons files, or give me some example commands you
have to run to get your code running?

Oh my goodness -- are you sure? Well, you seem like a brave fellow, so I've made a tarball with a dummy project which you can try out, with SCons. I've put it here. Follow the README and see what you can make of it. ;)

> So, a few questions:
>
> * What's the appropriate way to handle a raft of alternatively-named
> GCC and Binutils programs like this? I'm thinking of adding a "bin-
> prefix" option to the main GCC builder -- would this be appropriate?
> This could, I think, also serve the AVR builder, and there are other
> distributions of GCC which have the same property, like MSP430 GCC. I
> could submit a patch / fork if so.

I'm not sure if that's needed. Assuming your version of gcc acts just
like a vanilla version of gcc (ie, no extra flags), I think the best
solution would be to use the default_exes keyword argument. Since you
know you need to use gcc, I'm guessing you can start out with:

   import fbuild.builders.c.gcc

   def build(ctx):
       builder = fbuild.builders.c.gcc.static(ctx,
           default_exes=['rm-none-eabi-gcc'],
           cross_compiler=True)
       ...

Great -- looks like the default_exes is what I need.

As to options, though: if you look through the example project I made, you'll see that this environment definitely needs extra flags -- piles of them. Actually there are so many, and they're so esoteric, that I'd expect to simply supply them in the builder argument above.

I suppose that supplying compiler-specific arguments at the top level doesn't make the environment very portable across compilers, but on these projects I usually find it necessary, or at least desirable, to tweak the compiler to within an inch of its life, for example to save code space. And typically I pick one compiler and stick with it anyway, so compiler portability isn't a massive concern -- but it can be nice. Being able to compile in different OS environments is very nice, too.

By the way, since you're not focussed on embedded stuff yourself, are you really sure you want to dive into all this? It can be very messy. Not that I'll turn down your help of course :)

> * In the ARM7TDMI environment, it is usually necessary to compile
> different parts of the code with different options. I need to be able
> to compile a certain set of objects with one set of options, another
> set with different options, and finally link these objects together
> into the final ELF. How do I move these lists of objects around? The
> right way to handle this in fbuild isn't clear to me.

Yeah, the documentation needs more work, I'll make sure to add a
section describing a scenario like this. Assuming you've got a
builder, you should be able to do things like:

   objs1 = builder.build_objs(['a.c', 'b.c'], optimize=True,
macros=['FOO', 'BAR'])
   objs2 = builder.build_objs(['c.c', 'd.c'], debug=True, macros=['BAZ'])
   exe = builder.link_exe('exe', objs1 + objs2) 

Or you can configure two builders:

   builder1 = fbuild.builders.c.gcc.static(ctx, ..., optimize=True,
macros=['FOO', 'BAR'])
   objs1 = builder2.build_objs(['a.c', 'b.c'])

   builder2 = fbuild.builders.c.gcc.static(ctx, ..., debug=True,
macros=['BAZ'])
   objs2 = builder2.build_objs(['c.c', 'd.c'])

   builder3 = fbuild.builders.c.gcc.static(ctx, ...)
   exe = builder3.link_exe('exe', objs1 + objs2)

This is perfect -- exactly what I was looking for.

> * Less important to me but still interesting: Obviously the usual
> configuration tests won't work, but some of them might be useful -- it
> could be helpful to verify that the compiler at least runs. Where
> would be the right place to implement those tests? Would this work
> somehow with the cross_compiler option?

I wasn't sure how to test cross compiled executables in the general
case, so fbuild only tests if the compiler can build things, but it
doesn't actually make sure those things can execute.

That's plenty for me -- a simulator wouldn't be much benefit to me there. Just knowing that the compiler is installed is typically enough.

Thanks for trying it out! Please let me know if you have more questions.

Not yet -- moving along .. thanks for your very extensive answers.

--
Quidquid latine dictum sit, altum viditur.

[Erick Tryzelaar]

On Fri, Jan 28, 2011 at 12:56 AM, Marek Kubica <ma...@xivilization.net> wrote:
>
> Yes I'm around, but as that project is done, I haven't done anything
> with AVR since. Which is a shame, but I got quite busy with other stuff.

That's too bad :) Well let me know if you end up using it for any
other project and run into problems :)

[Erick Tryzelaar]

On Sat, Jan 29, 2011 at 12:16 AM, Michael Ashton <da...@gtf.org> wrote:
>
> Oh my goodness -- are you sure? Well, you seem like a brave fellow, so I've
> made a tarball with a dummy project which you can try out, with SCons. I've
> put it here. Follow the README and see what you can make of it. ;)

Of course! I'm crazy enough to write my own build system, going
through your setup doesn't seem so bad to me :) The link is not
working for me though, do you have to explicitly share it with me or
something?

> As to options, though: if you look through the example project I made,
> you'll see that this environment definitely needs extra flags -- piles of
> them. Actually there are so many, and they're so esoteric, that I'd expect
> to simply supply them in the builder argument above.
> I suppose that supplying compiler-specific arguments at the top level
> doesn't make the environment very portable across compilers, but on these
> projects I usually find it necessary, or at least desirable, to tweak the
> compiler to within an inch of its life, for example to save code space. And
> typically I pick one compiler and stick with it anyway, so compiler
> portability isn't a massive concern -- but it can be nice. Being able to
> compile in different OS environments is very nice, too.

I agree that it gets a little silly to expose all the compiler options
directly, so when you really want to get low level, there are cflags
(for compiler options) and lflags (for linker options). You can
specify them with:

builder.build_exe(...,
clags=[
'-g',
'-mno-int16',
'-mlibfuncs'],
lflags=[
'-nostdlib',
'shared-libgcc'])

Just be aware that there are a couple options fbuild reserves for
itself, like "-o", and things may error out or get confused if you
specify them as well.

> By the way, since you're not focussed on embedded stuff yourself, are you
> really sure you want to dive into all this? It can be very messy. Not that
> I'll turn down your help of course :)

It's really no problem at all :) We may have to play email tag to get
things working, but I'm more thankful that you're willing to try
fbuild :)

[Michael Ashton]

On Mon, Jan 31, 2011 at 1:39 PM, Erick Tryzelaar <erick.t...@gmail.com> wrote:

On Sat, Jan 29, 2011 at 12:16 AM, Michael Ashton <da...@gtf.org> wrote:
>
> Oh my goodness -- are you sure? Well, you seem like a brave fellow, so I've
> made a tarball with a dummy project which you can try out, with SCons. I've
> put it here. Follow the README and see what you can make of it. ;)

Of course! I'm crazy enough to write my own build system, going
through your setup doesn't seem so bad to me :) The link is not
working for me though, do you have to explicitly share it with me or
something?

I thought I did that .. try this one?

https://docs.google.com/leaf?id=0B_LBTHn5Wil4YWE3NThlYTEtOWFkNC00YjlkLTlkYzgtYTBiNTgyZDBkZDA3&hl=en&authkey=CLKh05kE

I'll send you something directly too. It's only 20k.

Sweet. Again, almost all of the options I'd want to use there are pretty esoteric, so I think that will work fine for starters.

 

> By the way, since you're not focussed on embedded stuff yourself, are you
> really sure you want to dive into all this? It can be very messy. Not that
> I'll turn down your help of course :)

It's really no problem at all :) We may have to play email tag to get
things working, but I'm more thankful that you're willing to try
fbuild :)

You're welcome -- if you can rescue me from SCons, I'll of course be indebted! --mpa 

 

[Erick Tryzelaar]

On Mon, Jan 31, 2011 at 4:18 PM, Michael Ashton <da...@gtf.org> wrote:
>
> On Mon, Jan 31, 2011 at 1:39 PM, Erick Tryzelaar <erick.t...@gmail.com>
> wrote:
>>
>> On Sat, Jan 29, 2011 at 12:16 AM, Michael Ashton <da...@gtf.org> wrote:
>> >
>> > Oh my goodness -- are you sure? Well, you seem like a brave fellow, so
>> > I've
>> > made a tarball with a dummy project which you can try out, with SCons.
>> > I've
>> > put it here. Follow the README and see what you can make of it. ;)
>>
>> Of course! I'm crazy enough to write my own build system, going
>> through your setup doesn't seem so bad to me :) The link is not
>> working for me though, do you have to explicitly share it with me or
>> something?
>
> I thought I did that .. try this one?
> https://docs.google.com/leaf?id=0B_LBTHn5Wil4YWE3NThlYTEtOWFkNC00YjlkLTlkYzgtYTBiNTgyZDBkZDA3&hl=en&authkey=CLKh05kE
>
> I'll send you something directly too. It's only 20k.

That one worked! I'll check it out.

> Sweet. Again, almost all of the options I'd want to use there are pretty
> esoteric, so I think that will work fine for starters.

Great, glad to hear it.

[Erick Tryzelaar]

Hey Michael! I'm still around, I've just been quite busy over the last
week or so. I ran into a problem trying to get the code sourcery
library to link a library. Is this not supported?

I've attached my fbuildroot.py script, and it's erroring out during
configuration with:

+ /scratch/foo/arm-2010.09/bin/arm-none-eabi-g++
-T/scratch/foo/arm_scons_example/build/aduc_example/ADuC7026-ROM.ld
-I/tmp/tmp3MxNMW -L/scratch/foo/arm-2010.09/lib -L/tmp/tmp3MxNMW -o
/tmp/tmp3MxNMW/tempexe /tmp/tmp3MxNMW/tempexe.o -ltemplib
/scratch/foo/arm-2010.09/bin/../lib/gcc/arm-none-eabi/4.5.1/../../../../arm-none-eabi/bin/ld:
warning: cannot find entry symbol _startup; defaulting to 00080000
/scratch/foo/arm-2010.09/bin/../lib/gcc/arm-none-eabi/4.5.1/libgcc.a(unwind-arm.o):
In function `get_eit_entry':
unwind-arm.c:(.text+0x204): undefined reference to `__exidx_start'
unwind-arm.c:(.text+0x208): undefined reference to `__exidx_end'
/scratch/foo/arm-2010.09/bin/../lib/gcc/arm-none-eabi/4.5.1/../../../../arm-none-eabi/lib/libc.a(lib_a-abort.o):
In function `abort':
abort.c:(.text+0x10): undefined reference to `_exit'
/scratch/foo/arm-2010.09/bin/../lib/gcc/arm-none-eabi/4.5.1/../../../../arm-none-eabi/lib/libc.a(lib_a-signalr.o):
In function `_kill_r':
signalr.c:(.text+0x1c): undefined reference to `_kill'
/scratch/foo/arm-2010.09/bin/../lib/gcc/arm-none-eabi/4.5.1/../../../../arm-none-eabi/lib/libc.a(lib_a-signalr.o):
In function `_getpid_r':
signalr.c:(.text+0x44): undefined reference to `_getpid'
/scratch/foo/arm-2010.09/bin/../lib/gcc/arm-none-eabi/4.5.1/../../../../arm-none-eabi/lib/libc.a(lib_a-sbrkr.o):
In function `_sbrk_r':
sbrkr.c:(.text+0x18): undefined reference to `_sbrk'
/scratch/foo/arm-2010.09/bin/../lib/gcc/arm-none-eabi/4.5.1/../../../../arm-none-eabi/lib/libc.a(lib_a-writer.o):
In function `_write_r':
writer.c:(.text+0x20): undefined reference to `_write'
/scratch/foo/arm-2010.09/bin/../lib/gcc/arm-none-eabi/4.5.1/../../../../arm-none-eabi/lib/libc.a(lib_a-closer.o):
In function `_close_r':
closer.c:(.text+0x18): undefined reference to `_close'
/scratch/foo/arm-2010.09/bin/../lib/gcc/arm-none-eabi/4.5.1/../../../../arm-none-eabi/lib/libc.a(lib_a-fstatr.o):
In function `_fstat_r':
fstatr.c:(.text+0x1c): undefined reference to `_fstat'
/scratch/foo/arm-2010.09/bin/../lib/gcc/arm-none-eabi/4.5.1/../../../../arm-none-eabi/lib/libc.a(lib_a-isattyr.o):
In function `_isatty_r':
isattyr.c:(.text+0x18): undefined reference to `_isatty'
/scratch/foo/arm-2010.09/bin/../lib/gcc/arm-none-eabi/4.5.1/../../../../arm-none-eabi/lib/libc.a(lib_a-lseekr.o):
In function `_lseek_r':
lseekr.c:(.text+0x20): undefined reference to `_lseek'
/scratch/foo/arm-2010.09/bin/../lib/gcc/arm-none-eabi/4.5.1/../../../../arm-none-eabi/lib/libc.a(lib_a-readr.o):
In function `_read_r':
readr.c:(.text+0x20): undefined reference to `_read'
collect2: ld returned 1 exit status
- exit 1, 0.02 sec

Do you know what's going on and how to fix it?

-------

import fbuild.builders.c.gcc
import fbuild.builders.cxx.gxx

def build(ctx):

#c_builder = fbuild.builders.c.gcc.static(ctx,
# default_exes=['arm-none-eabi-gcc'],
# libpaths=['/scratch/foo/arm-2010.09/lib'],
# link_flags=['-T/scratch/foo/arm_scons_example/build/aduc_example/ADuC7026-ROM.ld'],
# cross_compiler=True)

cxx_builder = fbuild.builders.cxx.gxx.static(ctx,
default_exes=['arm-none-eabi-g++'],
libpaths=['/scratch/foo/arm-2010.09/lib'],
link_flags=['-T/scratch/foo/arm_scons_example/build/aduc_example/ADuC7026-ROM.ld'],
cross_compiler=True)

includes = ['build/aduc_example', 'build/aduc_example/include']
macros = ['GCC_ARM7', 'VERSION=']
warnings = ['all', 'no-char-subscripts', 'error=return-type', 'no-reorder']

objs = []
objs.extend(cxx_builder.build_objects([
'build/aduc_example/main.cc',
'build/aduc_example/ADUC702X_UART.cc'],
flags=['-mthumb-interwork', '-mthumb', '-fno-rtti',
'-fno-exceptions', '-frepo'],
macros=macros + ['THUMB_INTERWORK'],
warnings=warnings,
includes=includes))

objs.append(c_builder('build/aduc_example/startup.S',
macros=macros,
includes=includes))

objs.append(cxx_builder.compile('build/aduc_example/irq.cc',
flags=['-mthumb-interwork', '-fno-rtti', '-fno-exceptions', '-frepo'],
macros=macros + ['THUMB_INTERWORK'],
warnings=warnings,
includes=includes))

elf = cxx_builder.link_exe('build/aduc_example.elf', objs,
flags=[
'-Wl,--gc-sections',
'-nostdlib',
'-lc',
'-lgcc',
'-Wl,-M',
'-Wl,-Map=aduc_example.map',
'-Tbuild/aduc_example/ADuC7026-ROM.ld'])

ctx.execute(['arm-none-eabi-g++', elf, '-O', 'ihex', ctx.buildroot
/ 'build/aduc_example.hex'])