New AVR-GCC 8 Toolchain from Atmel

[Massimo Banzi]

Hello all

I Just got this link from ATMEL http://distribute.atmel.no/tools/opensource/Atmel-AVR-GNU-Toolchain/3.4.3/ 

it’s GCC 4.8.1 with a bunch of bug fixes.

if somebody wants to jump on it and test it that would be awesome. 

Arduino tshirts and gadgets to whoever can test this compiler and figure out if it solves the long standing issue we’ve been having :)

happy hacking

m

 

-- 
Massimo Banzi <m.b...@arduino.cc>
CEO, Arduino SA

[Vasilis Georgitzikis]

which is?

[Matthijs Kooijman]

Hey folks,

FWIW, I've been running on a (self-compiled and patched) gcc 4.8 for a
few months now, without problems. Not sure if and how that helps,
though.

Gr.

Matthijs

[Federico Fissore]

It seems like it's time to roll in https://github.com/arduino/toolchain-avr

First thing is porting/splitting the atmel script [0] and see what happens.

[0]
http://distribute.atmel.no/tools/opensource/Atmel-AVR-GNU-Toolchain/3.4.3/build-avr8-gnu-toolchain-git.sh

[Britton Kerin]

On Thu, Jan 30, 2014 at 11:33 AM, Massimo Banzi <m.b...@arduino.cc> wrote:
> Hello all
>
> I Just got this link from ATMEL
> http://distribute.atmel.no/tools/opensource/Atmel-AVR-GNU-Toolchain/3.4.3/
>
> it's GCC 4.8.1 with a bunch of bug fixes.

Are these fixes going to make it back into GCC proper eventually or is that not
possible for some reason?

> if somebody wants to jump on it and test it that would be awesome.
>

> Arduino tshirts and gadgets to whoever can test this compiler and figure out
> if it solves the long standing issue we've been having :)

Is there a page somewhere that summarizes the issues that are believed
to be toolchain-related?

Thanks,
Britton

[Massimo Banzi]

I think Atmel pushes those patches upstream but let me check with them
to make sure

The toolchain issues have been discussed several times on this list, I
don't have a list handy but the archives would provide a clue

m

> --
> You received this message because you are subscribed to the Google Groups "Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.



--
Massimo Banzi <m.b...@arduino.cc>

[ODwyerPW]

It would be nice to see avg-gcc 4.8.1 and sam-gcc 4.8.3 both make it into Arduino IDE 1.5.X .

Having the latest toolchain sets for both architectures when the IDE leaves Beta would be nice.

 

[Massimo Banzi]

Well dear Arduino community, together we can do it.

ARM has made available their compiler for all three platforms and we
just need to make the AVR code compile.
Federico has prepared a repo for it..

let's do it

m

[William "Chops" Westfield]

> Is there a page somewhere that summarizes the issues that are believed
> to be toolchain-related?

There is https://github.com/arduino/Arduino/issues/1208
But it is mostly a list of things wrong in previous "new" compilers, rather than a list of issues in the currently used compiler.

> Massimo Banzi <m.b...@arduino.cc> wrote:
>>
>> I Just got this link from ATMEL
>> http://distribute.atmel.no/tools/opensource/Atmel-AVR-GNU-Toolchain/3.4.3/
>>
>> it's GCC 4.8.1 with a bunch of bug fixes.

>> if somebody wants to jump on it and test it that would be awesome.

Is that an official statement long the lines of "The official Arduino people have become enthusiastic about actually upgrading the IDE to a new compiler version"? Previous efforts toward a new compiler version have been mostly ignored, officially.

There is still the issue of coming up with parallel versions for Windows/Mac/Linux…

WestfW

[Paul Stoffregen]

On 01/31/2014 06:20 PM, William "Chops" Westfield wrote:
>> Is there a page somewhere that summarizes the issues that are believed
>> to be toolchain-related?
> There is https://github.com/arduino/Arduino/issues/1208
> But it is mostly a list of things wrong in previous "new" compilers, rather than a list of issues in the currently used compiler.

Here is the list of currently used (version 4.3.2) AVR toolchain
shortcomings I'm personally aware of:


1: When warnings are enabled from File > Preferences "Show verbose
output...", the C++ compiler will falsely print "only initialized
variables can be placed into program memory area" when a PROGMEM
variable is used. This is a harmless but annoying compiler bug.

2: When linker relaxation is used, in some rare cases the compiler will
attempt to incorrectly use RJMP instead of JMP in the vector table,
resulting in a "R_AVR_13_PCREL" compile error. I believe Arduino Mega
2560 is the only official Arduino board using this optimization. AFAIK,
this is the only 4.3.2 bug that actually causes the compiler to not
produce working code.

3: Newer AVR chips, particularly the AVR XMEGA line, are not supported.
No official Arduino boards use these newer chips, but this has been a
sore point for people wishing to create 3rd party hardware with newer
Atmel parts.

4: Inline heuristics are not very good, causing the compiler to
sometimes avoid automatically converting small functions to inline when
doing so would clearly be beneficial.

5: When compiling C++, many optimization opportunities are missed, even
cases that are well optimized when compiling C.

6: Newer language features & extensions, like C++11 and named address
spaces, aren't available.

7: Many users automatically assume the newer versions must be better.


In some sense, the biggest "problem" has been that version 4.3.2 was
actually very good, especially compared to avr-gcc 4.4 and part of the
4.5 series that introduced truly horrific C++ bugs. Later versions also
made incompatible changes, particularly with PROGMEM requiring "const",
which has traditionally created a disincentive to upgrade. Perhaps now
many sketches and libraries have updated, since a small fraction of
Arduino users have Linux distro-provided copies with the newer toolchain.

>
>
>> Massimo Banzi <m.b...@arduino.cc> wrote:
>>> I Just got this link from ATMEL
>>> http://distribute.atmel.no/tools/opensource/Atmel-AVR-GNU-Toolchain/3.4.3/
>>>
>>> it's GCC 4.8.1 with a bunch of bug fixes.
>>> if somebody wants to jump on it and test it that would be awesome.
> Is that an official statement long the lines of "The official Arduino people have become enthusiastic about actually upgrading the IDE to a new compiler version"? Previous efforts toward a new compiler version have been mostly ignored, officially.
>

> There is still the issue of coming up with parallel versions for Windows/Mac/Linux...
>
> WestfW
>
>

[Massimo Banzi]

Just a clarification….

Previous efforts haven’t been ignored but every time the request was brought up there were always multiple issues with more modern compilers that made it impossible.

Upgranding the toolchain has been requested a number of time and we would love to be able to do it.

m

 

-- 
Massimo Banzi <m.b...@arduino.cc>
CEO, Arduino SA

There is still the issue of coming up with parallel versions for Windows/Mac/Linux...

WestfW

[Paul Stoffregen]

On 02/01/2014 03:06 AM, Massimo Banzi wrote:

Just a clarification….

Previous efforts haven’t been ignored but every time the request was brought up there were always multiple issues with more modern compilers that made it impossible.

Even if 4.8.1 is as bug-free as 4.3.2 on C++, there will be pain caused by this upgrade.  Publishing it in 1.5.6 or 1.5.7 and keeping the old toolchain in 1.0.5 and 1.0.6 is probably the best opportunity to minimize user suffering.

Upgranding the toolchain has been requested a number of time and we would love to be able to do it.

I'm willing to help test it.  I have ready-to-use test hardware for about 40 widely used 3rd party Arduino libraries with Teensy 2.0, which is the same chip as Arduino Leonardo.

But I'm not looking to build the whole toolchain from source, times 4 systems.  When someone has compiled the 4 copies proposed for inclusion in Arduino, please let me know.  Actually, I'll probably do nearly all my testing on the Linux 64 bit copy, since that's my primary system, so even just that one would let me get started.

[Cristian Maglie]

In data giovedì 30 gennaio 2014 21:38:11, Matthijs Kooijman ha scritto:
> FWIW, I've been running on a (self-compiled and patched) gcc 4.8 for a
> few months now, without problems. Not sure if and how that helps,
> though.

Personally I'd like to know which patch you have applied.
I see that there are various "flavors" of the avr-libc, avr-gcc and binutils:

- The official GNU version on savannah
- The Atmel version (provided by Atmel through their website)
- Any version packaged in any distribution (Debian, Ubuntu, WinAVR, Crosspack
and Arduino too).

Everyone has its own set of patches. Probably the most interesting to look
into now are the latest from Atmel.

C

[Matthijs Kooijman]

Hi Christian,

On Mon, Feb 03, 2014 at 04:43:28PM +0100, Cristian Maglie wrote:
> In data giovedì 30 gennaio 2014 21:38:11, Matthijs Kooijman ha scritto:
> > FWIW, I've been running on a (self-compiled and patched) gcc 4.8 for a
> > few months now, without problems. Not sure if and how that helps,
> > though.
>
> Personally I'd like to know which patch you have applied.

I've used gcc 4.8-2 and avr-libc 1.8.0-4 from Debian, including their
patches. I added a patch to both to support the 256RFR2 chip, which I
need.

I'm running binutils from Debian without any local patches, since the
Debian version already included a patch for the 256RFR2 chip.

> I see that there are various "flavors" of the avr-libc, avr-gcc and binutils:
>
> - The official GNU version on savannah
> - The Atmel version (provided by Atmel through their website)
> - Any version packaged in any distribution (Debian, Ubuntu, WinAVR, Crosspack
> and Arduino too).
>
> Everyone has its own set of patches. Probably the most interesting to look
> into now are the latest from Atmel.

Indeed. I think that others often take their pick of patches from Atmel
as well.

I'd also expect that the patches from Atmel would quickly get committed
to the upstream repositories, but I'm not certain if Atmel is actively
pushing it or just waiting for others to pick up their patches.

However, taking the 256rfr2 support as an example: This device was
released by the end of 2012. Looking at the timestamps inside the patch
file of the Atmel toolchain 3.4.2 release, it seems they published
patches for this between january and may 2013. However, gcc trunk
doesn't include support for this device yet (interestingly enough,
avr-libc trunk does support it already). This seems to indicate to me
that Atmel isn't actively pushing their patches, but perhaps others know
more about this (perhaps there's mailinglist discussions about some of
these patches?).

Gr.

Matthijs

[Cristian Maglie]

In data sabato 1 febbraio 2014 16:29:13, Paul Stoffregen ha scritto:
> Even if 4.8.1 is as bug-free as 4.3.2 on C++, there will be pain caused
> by this upgrade. Publishing it in 1.5.6 or 1.5.7 and keeping the old
> toolchain in 1.0.5 and 1.0.6 is probably the best opportunity to
> minimize user suffering.

Agree.

> I'm willing to help test it. I have ready-to-use test hardware for
> about 40 widely used 3rd party Arduino libraries with Teensy 2.0, which
> is the same chip as Arduino Leonardo.

Atmel released a precompiled version here:

Windows: http://www.atmel.com/tools/ATMELAVRTOOLCHAINFORWINDOWS.aspx
Linux: http://www.atmel.com/tools/ATMELAVRTOOLCHAINFORLINUX.aspx

Federico F. is working on the build script for mac (and the other OSs too). If
everything goes well, we'll probably have a nightly-build for testing in a day
or two.

C

[Paul Stoffregen]

> Atmel released a precompiled version here:
>
> Windows: http://www.atmel.com/tools/ATMELAVRTOOLCHAINFORWINDOWS.aspx
> Linux: http://www.atmel.com/tools/ATMELAVRTOOLCHAINFORLINUX.aspx
>
> Federico F. is working on the build script for mac (and the other OSs too). If
> everything goes well, we'll probably have a nightly-build for testing in a day
> or two.

Great. I'll hold out a couple days, for a download that doesn't require
a myAtmel account....

[Federico Fissore]

Hi everyone

We've just released a nightly that includes the latest toolchain from atmel

http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-windows.zip
http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-macosx.zip
http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-linux32.tgz
http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-linux64.tgz

I must admin I've barely tested it: I've successfully uploaded a Blink
and an ASCIITable on both a Leonardo and an Uno.

Code to build the toolchain on linux32, linux64, mac32 and windows is
available on the atmel-3.4.3 branch at
https://github.com/arduino/toolchain-avr/
It downloads gcc/avr-libc/binutils from gnu and patches it with a patch
we've produced comparing the two source trees

Looking forward for your feedback

Best regards

Federico

[lestofante]

in the italian forum many use the latest ATMEL gcc is already in use, here the guide for win/linux in italian of the http://forum.arduino.cc/index.php?topic=96976.660
also on mac (from 3.4.2): http://forum.arduino.cc/index.php?topic=186435.0

those fix are needed;
https://github.com/arduino/Arduino/pull/1447
https://github.com/arduino/Arduino/pull/1448

there are a little fix to be done on the code,

[Federico Fissore]

lestofante, il 04/02/2014 11:16, ha scritto:
> in the italian forum many use the latest ATMEL gcc is already in use,
> here the guide for win/linux in italian of the
> http://forum.arduino.cc/index.php?topic=96976.660
> also on mac (from 3.4.2): http://forum.arduino.cc/index.php?topic=186435.0
>
> those fix are needed;
> https://github.com/arduino/Arduino/pull/1447
> https://github.com/arduino/Arduino/pull/1448
>
> there are a little fix to be done on the code,

Thank you lestofante

In case these fixes are needed even with atmel 3.4.3 (and most likely
they are) please, refer to ide branch
https://github.com/arduino/Arduino/tree/ide-1.5.x-avr-toolchain-gcc-4.8.1

Federico

[lestofante]

someone also made an english blog post http://www.open-electronics.org/arduino-full-memory-upgrade-to-the-last-atmel-toolchain-version/

these changere areneeded because it fix some deprecated code that have been removed

[Matthijs Kooijman]

Hey Federico,

> We've just released a nightly that includes the latest toolchain from atmel

Cool!

> http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-windows.zip
> http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-macosx.zip
> http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-linux32.tgz
> http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-linux64.tgz

I've just given the linux64 build a spin, seems to work like a charm.
One problem though: it seems the build I just downloaded still uses
avrdude-5.11, while the toolchain-avr repository has 6.0.1 (which breaks
my uploads).

Looks like the build doesn't automatically uses the latest version from
the toolchain-avr repository, but instead uses a (manual?) build from
downloads.arduino.cc, so perhaps someone needs to bump this.


I'm particularly interested because I'm working on the Pinoccio board
(http://pinocc.io), which uses the atmega256rfr2 chip which needs a new
and patched toolchain. So I'm glad you included the patches from Atmel,
which add support for our chip :-)

> I must admin I've barely tested it: I've successfully uploaded a
> Blink and an ASCIITable on both a Leonardo and an Uno.

I just tested our main sketch, which is quite non-trivial and includes
bitlash, SPI wifi code and a mesh networking stack, which (after getting
avrdude 6.0.1) compiled and works fine, at first glance.

We will likely be using this nightly build version for our internal
development, as well as suggesting our users to use it once we ship out
our first boards soon, which should attract a significant amount of
testing :-) If any issues arise, I'll be sure to send bug reports or
pullrequests.


Thanks,

Matthijs

[Federico Fissore]

Hi Matthijs

I'm glad it's working fine :)

Matthijs Kooijman, il 25/02/2014 17:03, ha scritto:
>
> I've just given the linux64 build a spin, seems to work like a charm.
> One problem though: it seems the build I just downloaded still uses
> avrdude-5.11, while the toolchain-avr repository has 6.0.1 (which breaks
> my uploads).
>
> Looks like the build doesn't automatically uses the latest version from
> the toolchain-avr repository, but instead uses a (manual?) build from
> downloads.arduino.cc, so perhaps someone needs to bump this.
>

Indeed it's manual. A internal continuous integration server
automatically builds the toolchain for each OS we support. The act of
moving it to downloads.arduino.cc is manual as it requires an update of
the SHA sums on the IDE.

We are migrating some servers and this is causing the delay. Hopefully
later today, the nightly build will be up to date

>
> If any issues arise, I'll be sure to send bug reports or
> pullrequests.
>

The more the better :)

Federico

[Federico Fissore]

Federico Fissore, il 26/02/2014 09:21, ha scritto:
>
> We are migrating some servers and this is causing the delay. Hopefully
> later today, the nightly build will be up to date
>

We are still having issues with the server in charge of building the
nightly, but I've uploaded the updated toolchain zip files and pushed
the updated SHAs on branch ide-1.5.x-avr-toolchain-gcc-4.8.1

https://github.com/arduino/Arduino/tree/ide-1.5.x-avr-toolchain-gcc-4.8.1

federico

[Matthijs Kooijman]

Hi Federico,

> http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-macosx.zip
It seems there is a problem with the OSX version, see:
https://github.com/arduino/Arduino/issues/1901#issuecomment-36254632

However, it appears it is not totally broken, AFAIK two of my collegues
installed the nightly build on OSX yesterday.

Gr.

Matthijs

[Federico Fissore]

Nightly build is partially restored: these builds

http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-windows.zip
http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-macosx.zip
http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-linux32.tgz
http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-linux64.tgz

now includes avrdude 6.0.1

federico

[Rich Obermeyer]

I have done a library for the arLCD and ezLCD display which uses various Arduino CPUs for driving low cost 2.6 to 4.3 inch high resolution TFT touch screen displays including touch screen processing and widget management with only the rx and tx serial port of the Arduino.
The question is how to know when to specify the SERIAL port or SERIAL1 port for use by the different libraries. Different boards have different requirements. I would rather not change the library every time a new board comes out to include that name. I also want to make it easy for the user of the library.

The current ezLCD3xx library includes a define check for
#if defined(__AVR_ATmega32U4__) || defined(CORE_TEENSY) || defined(__AVR_ATmega2560__)
#define arSERIAL SERIAL1
If it's an UNO and various copies I can't remember names of
#define arSERIAL SERIAL

I know theres lots of different board names out there to check for.
1. What would be a better way to always use SERIAL1 without singling out each board name directly when needed?
2. where would I find the names of all these boards to put in the library if I have to specify them?
3. Who picked a global variable name of PINK to use in the libraries? PINK is a standard color name. They should have used something like pin_K. Not pink for a global definition.
IMHO the users should have been more careful naming pins on large chips. BGAs will only make this worse.

Thanks for the help,
Rich

[Cristian Maglie]

Hi Rich,

In data domenica 2 marzo 2014 09:29:57, Rich Obermeyer ha scritto:
> The current ezLCD3xx library includes a define check for
> #if defined(__AVR_ATmega32U4__) || defined(CORE_TEENSY) ||
> defined(__AVR_ATmega2560__) #define arSERIAL SERIAL1
> If it's an UNO and various copies I can't remember names of
> #define arSERIAL SERIAL

from 1.5.6 (and from the still to be released 1.0.6) the SERIAL_* defines have
been introduced, this is an example with some explanations in comments:

https://github.com/arduino/Arduino/blob/ide-1.5.x/hardware/arduino/avr/variants/leonardo/pins_arduino.h#L341

> I know theres lots of different board names out there to check for.
> 1. What would be a better way to always use SERIAL1 without singling out
> each board name directly when needed? 2. where would I find the names of
> all these boards to put in the library if I have to specify them?

in general every board should provide information about serial ports (and
everything that is hardware-dependent) through their variant file, otherwise is
up to you to find a way to discriminate the boards you're interested.
From the IDE 1.5.x we introduced a new property in the boards.txt file:

boards.id=BOARDNAME

this is used to create #defines to be checked in user code, for example the
previos property may translate into the following compile flag:

-DARDUINO_AVR_BOARDNAME

but I suggest to use it as a last chance.

> 3. Who
> picked a global variable name of PINK to use in the libraries? PINK is a
> standard color name. They should have used something like pin_K. Not pink
> for a global definition. IMHO the users should have been more careful
> naming pins on large chips. BGAs will only make this worse.

PINK is defined into the standard AVR library as:

avr/lib/avr/include/avr/iomxx0_1.h:
# define PINK _SFR_MEM8(0x106)

we cannot do anything for it (and probably changing it now is the wrong thing
to do).

C

PS: Please don't hijack threads: start a new conversation instead of changing
the subject, otherwise your message may pass unnoticed.

[Rich Obermeyer]

What I get from this 

An Arduino compatible shield that uses one of the serial ports must include in the library a check for every board they want to attach to.  Currently that's over 20 boards in 1.5.5-r2.  This also would mean every time a new board is designed or announced, the library would have to be modified to support that board unless the board is called the same as a previous board. 

 

One of the primary draw for users of Arduino related hardware/Software is the ease of use to the novice programmer.

Since I see new clone boards all the time that use different names than Arduino provides, this seems like a significant burden to the novice user that will not be able to figure out why a year old LCD driver does not work out of the box.

There has to be a better way.

--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

--
Rich

[Paul Stoffregen]

On 03/03/2014 02:19 PM, Rich Obermeyer wrote:

What I get from this 

An Arduino compatible shield that uses one of the serial ports must include in the library a check for every board they want to attach to.  Currently that's over 20 boards in 1.5.5-r2.  This also would mean every time a new board is designed or announced, the library would have to be modified to support that board unless the board is called the same as a previous board.

That's *exactly* why the SERIAL_XXX macros were added in 1.5.6 and will be in the upcoming 1.0.6 release.

The feature you're asking for has already been implemented.

One of the primary draw for users of Arduino related hardware/Software is the ease of use to the novice programmer.

Since I see new clone boards all the time that use different names than Arduino provides, this seems like a significant burden to the novice user that will not be able to figure out why a year old LCD driver does not work out of the box.

There has to be a better way.

Yes, that better way is already in 1.5.6.  All you have to do is use it.

You can even fall back automatically to a list of known boards when your library is used on an older version that doesn't have these.


For example, arLCD might use:

#if defined(SERIAL_PORT_HARDWARE_OPEN)
  // this one is probably the best default
  #define arSERIAL SERIAL_PORT_HARDWARE_OPEN

#elif defined(SERIAL_PORT_HARDWARE)
  // if none is open...
  #define arSERIAL SERIAL_PORT_HARDWARE

#elif defined(__AVR_ATmega32U4__) || defined(CORE_TEENSY) || defined(__AVR_ATmega2560__)
  // compiling on an old Arduino IDE with a board known to have Serial1
  #define arSERIAL Serial1

#else
  // final fallback....
  #define arSERIAL Serial

#endif


This stuff is documented in the comments within the variants/*/pins_arduino.h file.  Here's a copy (from the Leonardo variant), to save you a couple minutes looking it up:


// These serial port names are intended to allow libraries and architecture-neutral
// sketches to automatically default to the correct port name for a particular type
// of use.  For example, a GPS module would normally connect to SERIAL_PORT_HARDWARE_OPEN,
// the first hardware serial port whose RX/TX pins are not dedicated to another use.
//
// SERIAL_PORT_MONITOR        Port which normally prints to the Arduino Serial Monitor
//
// SERIAL_PORT_USBVIRTUAL     Port which is USB virtual serial
//
// SERIAL_PORT_LINUXBRIDGE    Port which connects to a Linux system via Bridge library
//
// SERIAL_PORT_HARDWARE       Hardware serial port, physical RX & TX pins.
//
// SERIAL_PORT_HARDWARE_OPEN  Hardware serial ports which are open for use.  Their RX & TX
//                            pins are NOT connected to anything by default.
#define SERIAL_PORT_MONITOR        Serial
#define SERIAL_PORT_USBVIRTUAL     Serial
#define SERIAL_PORT_HARDWARE       Serial1
#define SERIAL_PORT_HARDWARE_OPEN  Serial1

[Rich Obermeyer]

Thanks Paul

This makes a lot more sense.

[Cristian Maglie]

In data domenica 2 marzo 2014 07:55:07, Federico Fissore ha scritto:
> Nightly build is partially restored: these builds
>
> http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-
windows.zip
> http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-
macosx.zip
> http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-
linux32.tgz
> http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-
linux64.tgz
>
> now includes avrdude 6.0.1
>
> federico

Hi guys,

Does anyone else had the time to test the new toolchain?

I'm tempted to merge the updated gcc in the coming official releases, but I
really need more testing reports, even a simple "I tested the new toolchain
with my XXX project that uses YYY hardware, and it works ok" will makes me
more confident in adopting it.

Thanks!

C

[darryl piper]

4.8.0 up to 4.8.2 have a bug in code produced. 

GCC Bugzilla ref Id of 60486 has fuller details

[Matthijs Kooijman]

On Thu, Apr 03, 2014 at 11:12:20AM +0100, darryl piper wrote:
> 4.8.0 up to 4.8.2 have a bug in code produced.
>
> GCC Bugzilla ref Id of 60486 has fuller details

Bug is here: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=60486

It looks like the bug is a missed optimization in some cases, but
generates wrong code in other cases. Not sure how often this happens,
but this could very well be a blocker.

The bug is fixed in gcc master and should be released in 4.8.3, but
I could not find any definitive word on when that's planned.

Perhaps we should backport the fix into 4.8.2?



Regarding testing - We've been using the 4.8.2 nightlies for our
Pinoccio project for a while internally in our team. Right now, we're
halfway through shipping the first 1000 starter kits to our users, whom
we're also recommending to use the 4.8.2 nightlies (though a lot of them
will just use the default sketch and not touch the IDE at all). Still,
we've seen no reports of compiler related problems yet.


Gr.

Matthijs

[darryl piper]

I've patched and compiled my own 4.8.2 for 32bit Linux and windows. I don't have compiled binaries for Mac tho.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAlM9L9QACgkQz0nQ5oovr7wSKACggdgvD1SUiwxixuiFWjWMKwin
fbQAn20+LwGUp/pFG6WY/PC3HnwLFTuE
=xzvA
-----END PGP SIGNATURE-----

[Cristian Maglie]

Hi Darryl,

thanks for the valuable info!

May you like to produce a pull request for the gcc build here?

https://github.com/arduino/toolchain-avr

If you don't want, can you just give me a link to the patch so we can apply by
ourself to our build?

C

--
Cristian

[darryl piper]

diff -Nrcbpad gcc-4.8.(012)/gcc/config/avr/avr.c gcc-4.8.2-fixed/gcc/config/avr/avr.c
*** gcc-4.8.(012)/gcc/config/avr/avr.c    2014-03-14 09:27:20.000000000 +0000
--- gcc-4.8.2-fixed/gcc/config/avr/avr.c    2014-03-27 11:09:54.096568600 +0000
*************** avr_out_plus (rtx insn, rtx, int, int,
*** 6807,6812 ****
  /* Work out the shortest sequence.  */

!  avr_out_plus_1 (op, &len_minus, MINUS, &cc_plus, code_sat, sign, out_label);
!  avr_out_plus_1 (op, &len_plus, PLUS, &cc_minus, code_sat, sign, out_label);


--- 6807,6812 ----
    /* Work out the shortest sequence.  */

!  avr_out_plus_1 (op, &len_minus, MINUS, &cc_minus, code_sat, sign, out_label);
!  avr_out_plus_1 (op, &len_plus, PLUS, &cc_plus, code_sat, sign, out_label);


----------------------------

is what i used to patch my source.

--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

--
--
 Darryl

[Paul Stoffregen]

On 04/03/2014 02:53 AM, Cristian Maglie wrote:
> Does anyone else had the time to test the new toolchain? I'm tempted
> to merge the updated gcc in the coming official releases, but I really
> need more testing reports, even a simple "I tested the new toolchain
> with my XXX project that uses YYY hardware, and it works ok" will
> makes me more confident in adopting it. Thanks! C

I tested 4 widely used 3rd party libraries today. All seem to work
fine. The new toolchain builds smaller code, but sometimes results in
more RAM used. All 4 tests seemed to work well. I only checked
examples for basic functionality. I did not attempt to measure speed or
performance, other than noting memory usage.

All tests performed on a Teensy 2.0 (same AVR chip as Arduino Leonardo),
with Arduino 1.0.5 and Teensyduino 1.18 running on Linux 64 bit (Ubunut
12.04). I did not test the Windows or Mac versions.

I did NOT test AVRDUDE or the rest of the 1.5.x branch. To test the
toolchain, I replaced hardware/tools/avr in 1.0.5 with the new toolchain.

Here are the 4 tests I tried.

USB Host Shield : board_qc
gcc432 : code=13036, ram=208 : tests pass, usb device enumerates
gcc481 : code=11456, ram=208 : tests pass, usb device enumerates

Encoder Library : TwoKnobs
gcc432 : code=5544 , ram=163 : works, both knobs respond properly
gcc481 : code=4768 , ram=167 : works, both knobs respond properly

PS2 Keyboard Library : Simple_Test
gcc432 : code=6310 , ram=191 : works, typing on keyboard shows in
serial monitor
gcc481 : code=5204 , ram=251 : works, typing on keyboard shows in
serial monitor

Adafruit CC3000 : buildtest
gcc432 : code=21026 , ram=642 : ok, free ram 1902, init cc3000, fw
1.19, mac addr, finds local SSIDs, DHCP gets IP#, www.adafruit.com is
207.58.139.247, ping success
gcc481 : code=19312 , ram=650 : ok, free ram 1894, init cc3000, fw
1.19, mac addr, finds local SSIDs, DHCP gets IP#, www.adafruit.com is
207.58.139.247, ping success
In verbose mode, many compiler warnings are printed for Adafruit's
CC3000 library!

[Paul Stoffregen]

On 04/03/2014 02:53 AM, Cristian Maglie wrote:

> Does anyone else had the time to test the new toolchain?

I ran two more tests to benchmark compiled code performance. All tests
on Teensy 2.0, Arduino 1.0.5 w/ Teensyduino 1.18 on Linux 64 bit.

TL;DR = 4.8.1 appears to generate slightly faster code, but the
performance gain is minimal.

Benchmark_USB_Receive_Standard: (measures USB stack speed limited by
Serial.available & Serial.read processing 1 byte at a time)
gcc432 : code=3186 , ram=50 : Average bytes per second = 182080
gcc481 : code=3102 , ram=50 : Average bytes per second = 188585

Benchmark_USB_Receive_ReadBytes_Overhead: (measures USB stack speed
using Serial.readBytes, but limited by fixed delays to "use" the
received data)
gcc432 : code=3622 , ram=50 : Average bytes per second = 499994
gcc481 : code=3528 , ram=50 : Average bytes per second = 500034

[Federico Fissore]

Sorry for the delay: I forgot to send this email

Nightly builds are ready and now include patched toolchain

Patch is here
https://github.com/arduino/toolchain-avr/commit/5d8e0f5450f9976f582f8aa98926e8e77a0c161f

Nightly builds are here

http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-windows.zip
http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-macosx.zip
http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-linux32.tgz
http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-linux64.tgz

Any feedback is appreciated

Federico

[Paul Stoffregen]

On 04/06/2014 12:20 PM, Federico Fissore wrote:
>
> Any feedback is appreciated

Did you see the 6 tests I ran last Thursday (April 3)? Those included 4
third party libraries with a broad range of coding styles, and a huge
amount of actual source code. I also ran 2 performance tests that
should be quite sensitive to the code generated by the compiler.

If I'd known you were about to apply another patch, I would have waited
a few days.

Are you planning to upgrade to 4.8.2 or apply more patches? I can run
the tests all over again, and perhaps even more, but it does take some
work to set up the hardware for each test. I'd really prefer to run
them only 1 more time on the final version intended for an Arduino 1.5.X
release.

[Federico Fissore]

Paul Stoffregen, il 06/04/2014 21:32, ha scritto:
> On 04/06/2014 12:20 PM, Federico Fissore wrote:
>>
>> Any feedback is appreciated
>
> Did you see the 6 tests I ran last Thursday (April 3)? Those included 4
> third party libraries with a broad range of coding styles, and a huge
> amount of actual source code. I also ran 2 performance tests that
> should be quite sensitive to the code generated by the compiler.
>

I've seen them and they look really good. Thank you

> If I'd known you were about to apply another patch, I would have waited
> a few days.

I'm really sorry for the misunderstanding. I thought Darryl's patch was
to be considered critical as it lead to wrong code and immediately
started the build/packing process to deliver it. I didn't mean to void
your test results.

>
> Are you planning to upgrade to 4.8.2 or apply more patches? I can run
> the tests all over again, and perhaps even more, but it does take some
> work to set up the hardware for each test. I'd really prefer to run
> them only 1 more time on the final version intended for an Arduino 1.5.X
> release.
>

No, we won't upgrade to 4.8.2, not until Atmel releases a newer
toolchain. No more patches are planned, neither Darryl's was. If we find
a bug and build/find its patch, we'll need to apply it. However, a brief
search on gcc bugzilla shows no pending bugs*

* actually one, http://gcc.gnu.org/bugzilla/show_bug.cgi?id=52278. Given
its age, I don't see a patch coming in the near future.

Federico

[Cristian Maglie]

Hi Paul,

In data domenica 6 aprile 2014 21:32:55, Paul Stoffregen ha scritto:
> Did you see the 6 tests I ran last Thursday (April 3)?

Yes, really appreciated!

> If I'd known you were about to apply another patch, I would have waited
> a few days.

You're right, sorry about that, we didn't mean to waste your time. Timings was
a bit unlucky and, of course, we always have room for improvements in
communicating what's going on.

C

[Paul Stoffregen]

I'll re-run those tests and several more later this week....

[Massimo Banzi]

thanks Paul!

hopefully this new gcc will solve a lot of the outstanding issues people have been having :)

m

-- 
Massimo Banzi <m.b...@arduino.cc>
CEO, Arduino SA

[David Farning]

On Monday, April 7, 2014 9:55:10 AM UTC-5, Massimo Banzi wrote:

thanks Paul!

hopefully this new gcc will solve a lot of the outstanding issues people have been having :)

m

Not sure if this helps....

I just did the set of experiments included in the SparkFun Inventor's Kit for Arduino using the IDE at 

http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-linux64.tgz downloaded on April 19 without any errors.

David Farning

  

-- 
Massimo Banzi <m.b...@arduino.cc>
CEO, Arduino SA

On 7 April 2014 at 10:18:21, Paul Stoffregen (pa...@pjrc.com) wrote:

I'll re-run those tests and several more later this week....


On 04/07/2014 02:29 AM, Cristian Maglie wrote:
> Hi Paul,
>
> In data domenica 6 aprile 2014 21:32:55, Paul Stoffregen ha scritto:
>> Did you see the 6 tests I ran last Thursday (April 3)?
> Yes, really appreciated!
>
>> If I'd known you were about to apply another patch, I would have waited
>> a few days.
> You're right, sorry about that, we didn't mean to waste your time. Timings was
> a bit unlucky and, of course, we always have room for improvements in
> communicating what's going on.
>
> C
>

--
You received this message because you are subscribed to the Google Groups "Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@arduino.cc.

[Federico Fissore]

David Farning, il 19/04/2014 21:09, ha scritto:
>
> Not sure if this helps....
>
> I just did the set of experiments included in the SparkFun Inventor's
> Kit for Arduino using the IDE at
> http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-linux64.tgz downloaded
> on April 19 without any errors.
>
> David Farning
>

Good news, thank you David :)

[Ralph Doncaster]

Just tried the windows version on 7E-64bit.

With a basic blink LED sketch with the Uno as a target, size was 1062bytes vs 1084 on 1.0.5-r2.

Link-time optimization (-flto) made a much bigger difference: 908 bytes.

There was a bunch of

cygwin warning:

  MS-DOS style path detected: sketch_apr22a.cpp

That can be fixed by setting the cygwin environment variable nodosfilewarning.

On Tuesday, 4 February 2014 06:10:56 UTC-4, f.fissore wrote:

Hi everyone

We've just released a nightly that includes the latest toolchain from atmel

http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-windows.zip
http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-macosx.zip
http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-linux32.tgz
http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-linux64.tgz

I must admin I've barely tested it: I've successfully uploaded a Blink
and an ASCIITable on both a Leonardo and an Uno.

Code to build the toolchain on linux32, linux64, mac32 and windows is
available on the atmel-3.4.3 branch at
https://github.com/arduino/toolchain-avr/
It downloads gcc/avr-libc/binutils from gnu and patches it with a patch
we've produced comparing the two source trees

Looking forward for your feedback

Best regards

Federico

Paul Stoffregen, il 03/02/2014 17:40, ha scritto:
>
>> Atmel released a precompiled version here:
>>
>> Windows: http://www.atmel.com/tools/ATMELAVRTOOLCHAINFORWINDOWS.aspx
>> Linux: http://www.atmel.com/tools/ATMELAVRTOOLCHAINFORLINUX.aspx
>>
>> Federico F. is working on the build script for mac (and the other OSs
>> too). If
>> everything goes well, we'll probably have a nightly-build for testing
>> in a day
>> or two.
>
> Great.  I'll hold out a couple days, for a download that doesn't require
> a myAtmel account....
>

[Ralph Doncaster]

I've done some more tests with LTO, and not only does it reduce the code size, I think it provides a solution to the slow digitalWrite problem.

http://nerdralph.blogspot.ca/2014/04/gcc-link-time-optimization-can-fix-bad.html

http://nerdralph.blogspot.ca/2014/04/a-better-digitalwrite-for-arduino.html

[Federico Fissore]

Thank you Ralph

Beginning next week, I'll update the IDE so that it will skip the warnings

I'm also positive in adding the LTO flag.

Has anyone tried this flag? Are there known reasons for not using it?

Federico

Ralph Doncaster, il 25/04/2014 23:13, ha scritto:
> I've done some more tests with LTO, and not only does it reduce the code
> size, I think it provides a solution to the slow digitalWrite problem.
> http://nerdralph.blogspot.ca/2014/04/gcc-link-time-optimization-can-fix-bad.html
> http://nerdralph.blogspot.ca/2014/04/a-better-digitalwrite-for-arduino.html
>
> On Tuesday, 22 April 2014 14:33:45 UTC-3, Ralph Doncaster wrote:
>
> Just tried the windows version on 7E-64bit.
> With a basic blink LED sketch with the Uno as a target, size
> was 1062bytes vs 1084 on 1.0.5-r2.
> Link-time optimization (-flto) made a much bigger difference: 908 bytes.
>
> There was a bunch of
> cygwin warning:
> MS-DOS style path detected: sketch_apr22a.cpp
>
> That can be fixed by setting the cygwin environment variable
> nodosfilewarning.
>
> On Tuesday, 4 February 2014 06:10:56 UTC-4, f.fissore wrote:
>
> Hi everyone
>
> We've just released a nightly that includes the latest toolchain
> from atmel
>
> http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-windows.zip

> <http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-windows.zip>
>
> http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-macosx.zip
> <http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-macosx.zip>
>
> http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-linux32.tgz
> <http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-linux32.tgz>
>
> http://downloads.arduino.cc/arduino-avr-toolchain-nightly-gcc-4.8.1-linux64.tgz

> --
> You received this message because you are subscribed to the Google
> Groups "Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to developers+...@arduino.cc
> <mailto:developers+...@arduino.cc>.

[Mikael Patel]

I have been using the updated AVR tool chain since little over a month
back for Cosa development. Overall it works fine. So far these are my
findings;

1. Parameter evaluation order
The new optimizer will somethings change evaluation order of function
parameter expressions. Do not use any side effects between parameter
expressions which rely on order.

2. Inline
With -Os the compiler will not inline as much as previously. To solve
this I used the function __attribute__((always_inline)) which has the
side-effect that the Cosa Pin operations became even faster.

3. Link time optimization
Did some tests with LTO especially for Cosa support for ATtinyXX. There
was a positive reduction in footprint. Usage of flash was reduced
(10-15%) but there is also an increase in SRAM usage. On ATtiny this is
an issue. I will do some more testing of this.

4. Remove unreferenced functions and data
Some of the Cosa example and test sketches became larger and exceeded
the ATtinyXX memory space. When looking at the link map I found a number
of unreferenced functions and data that previously where removed. This
can be solved on file level by moving the "cluster" into a separate file.

5. Compiler Warnings
For Cosa build I have added -Wall and tested -W and -pedantic. I would
recommend this as this helps find issues.

Cheers!

Mikael

[Matthijs Kooijman]

Hey folks,

> >I've done some more tests with LTO, and not only does it reduce the code
> >size, I think it provides a solution to the slow digitalWrite problem.
> >http://nerdralph.blogspot.ca/2014/04/gcc-link-time-optimization-can-fix-bad.html
> >http://nerdralph.blogspot.ca/2014/04/a-better-digitalwrite-for-arduino.html

If I read your posts correctly, you are saying that lto will allow the
compile to better detect that certain values (global variables mostly)
are never changed and thus effectively constants. After figuring that
out, other (normal) optimizations can be used to generate more efficient
code.

Furthermore, if the global variable would be properly defined as
"const", or an actual constant would be used ("digitalWrite(2, HIGH)"),
LTO does not improve things (in this regard, of course LTO opens up
other optimization opportunities as well).

So, you're saying that LTO in this way allows generating better code,
even when users aren't using the "const" keyword as consistent as they
could/should do, hence it can be an efficiency gain for novice
programmers.

Furthermore, I think there is another gain here (which you might have
already realized): You can make a generic library that _allows_
things to be variable (e.g. has a global non-const pin number variable
that can be changed by the sketch). However, if the sketch doesn't
actually change the variable (which is not known before link time), lto
allows the compiler to generate smarter code taking this into account.

In any case, I agree that enabling lto would be awesome!

> I'm also positive in adding the LTO flag.
>
> Has anyone tried this flag? Are there known reasons for not using it?

I tried a while back, but ran into the problem that avr-gcc doesn't have
a "linker plugin", which would allow the compiler to look inside
libraries to see what symbols they use. In the Arduino case, all of the
core is compiled into a core.a library file, which is then included in
the link. Because lto doesn't know how to look inside the core.a file,
it does not know that the loop() and main() functions are referenced
from inside core.a and will thus optimize them away (w00ps!).

We can fix this by explicitely telling the compiler that loop() and
main() are important, but I think the other effect of this is that the
code in core.a does not get lto applied, so all these optimizations
affect the sketch and libraries, but not the Arduino core (which would
be a pity!).

Note that this also involves passing -fwhole-program, perhaps doing lto
without -fwhole-program can still provide some of the lto benefits,
without breaking the build. Or perhaps avr-gcc 4.8 includes the linker
plugin, since I tested with 4.7.

More detailed analysis (and also the issue to track lto support) is at
https://github.com/arduino/Arduino/issues/660

Ralph suggests that he has used an actual Blink sketch with the Arduino
code, so perhaps he has found a way around this? Or perhaps he is using
avr-gcc directly and skips the core.a step, including the core .o file
directly in the build?

Gr.

Matthijs

[Ralph Doncaster]

On Saturday, 26 April 2014 06:31:45 UTC-3, Matthijs Kooijman wrote:

So, you're saying that LTO in this way allows generating better code,
even when users aren't using the "const" keyword as consistent as they
could/should do, hence it can be an efficiency gain for novice
programmers.

Yes, and even better is it should work when you can't practically make something const - like the issues you might encounter changing an instance of an object to const, but some methods that do not modify state are not declared const.

I see const as being an overloaded keyword - one meaning is that I want to protect this object from being changed by other code, and the other meaning is essentially a compiler hint to say the object wont' be changed.   With LTO there is no need for the second meaning, which IMHO makes things a bit cleaner.

Note that this also involves passing -fwhole-program, perhaps doing lto
without -fwhole-program can still provide some of the lto benefits,
without breaking the build. Or perhaps avr-gcc 4.8 includes the linker
plugin, since I tested with 4.7.

More detailed analysis (and also the issue to track lto support) is at
https://github.com/arduino/Arduino/issues/660

Ralph suggests that he has used an actual Blink sketch with the Arduino
code, so perhaps he has found a way around this? Or perhaps he is using
avr-gcc directly and skips the core.a step, including the core .o file
directly in the build?

It would seem things have changed from 4.7.

Although I have tested avr-gcc 4.8 (MingW and Linux-x64), the blink sketch test was done with the Arduino 4.8.1 nightly (screen shot attached).  I didn't use -fwhole-program; the gnu docs say, "WHOPR stands for WHOle Program optimizeR (not to be confused with the semantics of -fwhole-program)"   The only thing I changed was platform.txt to add -flto.  The flags lines changed are: compiler.c, .c.elf, .cpp, and .ldflags.

I'm confused about your comment about core.a.  Are you saying the compiler flags to build the core are different than those in platform.txt?  If so, then there should be further size reductions with building the core with LTO.

Another example of code size reductions is VirtualWire.  Building the VirtualWire-1.25 transmitter sketch with lto gives:

Sketch uses 1,974 bytes (6%) of program storage space. Maximum is 32,256 bytes.

Global variables use 222 bytes (10%) of dynamic memory, leaving 1,826 bytes for local variables. Maximum is 2,048 bytes.

Without lto: (49.5% more flash, 44% more ram)

Sketch uses 2,932 bytes (9%) of program storage space. Maximum is 32,256 bytes.

Global variables use 320 byte

This is just the build results - I still have to verify that it still works correctly.

[Ralph Doncaster]

On Saturday, 26 April 2014 06:29:06 UTC-3, Mikael Patel wrote:

With -Os the compiler will not inline as much as previously. To solve
this I used the function __attribute__((always_inline)) which has the
side-effect that the Cosa Pin operations became even faster.

And the code is no larger than without always_inline?  And was this with -flto?

 

[Matthijs Kooijman]

Hey Ralph,

> The only thing I changed was platform.txt to add -flto. The flags lines
> changed are: compiler.c, .c.elf, .cpp, and .ldflags.
>
> I'm confused about your comment about core.a. Are you saying the compiler
> flags to build the core are different than those in platform.txt? If so,
> then there should be further size reductions with building the core with
> LTO.

That's not exactly what I'm saying. It's not the flag that's different,
but the route from source to linker.

Source files in the sketch and libraries are compiled to .o files. Then,
during linking, all those .o files are passed to the linker.

Source files in the core are first compiled to .o files and then
collected in a .a file (archive) using the avr-ar command. Then the .a
file is passed to the linker.

Now, the problem was that the lto could not access the partly compiled
lto "code" inside .a files, which would mean the efficiency of lto was
greatly reduced. AFAIU, getting at the code from .a files normally
requires a "linker plugin", which wasn't available in avr-gcc 4.7. It
seems this was fixed in avr-gcc .8, I now have this file:
/usr/lib/gcc/avr/4.8.2/liblto_plugin.so.0

With the linker plugin, the .a stuff doesn't seem relevant anymore, gcc
can just do the right thing even with .a files.

This should help a lot to get things to compile smaller and faster. I'll
certainly be enabling -flto here :-D

Gr.

Matthijs

[Ralph Doncaster]

Thanks for the explanation.  I'm not great at following build steps since a makefile is not a linear flow of execution.

On Saturday, 26 April 2014 12:46:28 UTC-3, Matthijs Kooijman wrote:

Hey Ralph,

[Federico Fissore]

Ralph Doncaster, il 22/04/2014 19:33, ha scritto:
> Just tried the windows version on 7E-64bit.
> With a basic blink LED sketch with the Uno as a target, size
> was 1062bytes vs 1084 on 1.0.5-r2.
> Link-time optimization (-flto) made a much bigger difference: 908 bytes.
>
> There was a bunch of
> cygwin warning:
> MS-DOS style path detected: sketch_apr22a.cpp
>
> That can be fixed by setting the cygwin environment variable
> nodosfilewarning.

Ralph, I've just tried running the latest nightly on a W7pro64, and I
see no cygwing warnings
Is there some additional detail that I'm missing?

federico

[Steve Harris]

Hi all,

I've been lurking for a while but wanted to report back on a test!

On Saturday, 26 April 2014 15:51:59 UTC+1, Ralph Doncaster wrote:

Although I have tested avr-gcc 4.8 (MingW and Linux-x64), the blink sketch test was done with the Arduino 4.8.1 nightly (screen shot attached).  I didn't use -fwhole-program; the gnu docs say, "WHOPR stands for WHOle Program optimizeR (not to be confused with the semantics of -fwhole-program)"   The only thing I changed was platform.txt to add -flto.  The flags lines changed are: compiler.c, .c.elf, .cpp, and .ldflags.

I've just tried compiling my Mega2560 program with the latest nightly and these -flto flags, and I get these warnings:

/cygdrive/c/Users/steve/AppData/Local/Temp/ccfQ5pcg.s: Assembler messages:
/cygdrive/c/Users/steve/AppData/Local/Temp/ccfQ5pcg.s:1754: Warning: ignoring changed section attributes for .progmem.data
/cygdrive/c/Users/steve/AppData/Local/Temp/ccfQ5pcg.s:1778: Warning: ignoring changed section attributes for .progmem.data
/cygdrive/c/Users/steve/AppData/Local/Temp/ccvten6c.s: Assembler messages:
/cygdrive/c/Users/steve/AppData/Local/Temp/ccvten6c.s:1626: Warning: ignoring changed section attributes for .progmem.data
In function '__vector_23':
C:\Temp\arduino\arduinoAvrToolchainNightlyGcc481\hardware\arduino\avr\cores\arduino\wiring.c:47:1: warning: '_vector_23' appears to be a misspelled signal handler [enabled by default]
 ISR(TIMER0_OVF_vect)
 ^
In function '__vector_25':
C:\Temp\arduino\arduinoAvrToolchainNightlyGcc481\hardware\arduino\avr\cores\arduino\HardwareSerial0.cpp:42:3: warning: '_vector_25' appears to be a misspelled signal handler [enabled by default]
   ISR(USART0_RX_vect)
   ^
In function '__vector_26':
C:\Temp\arduino\arduinoAvrToolchainNightlyGcc481\hardware\arduino\avr\cores\arduino\HardwareSerial0.cpp:57:1: warning: '_vector_26' appears to be a misspelled signal handler [enabled by default]
 ISR(USART0_UDRE_vect)
 ^
In function '__vector_36':
C:\Temp\arduino\arduinoAvrToolchainNightlyGcc481\hardware\arduino\avr\cores\arduino\HardwareSerial1.cpp:42:3: warning: '_vector_36' appears to be a misspelled signal handler [enabled by default]
   ISR(USART1_RX_vect)
   ^
In function '__vector_37':
C:\Temp\arduino\arduinoAvrToolchainNightlyGcc481\hardware\arduino\avr\cores\arduino\HardwareSerial1.cpp:57:1: warning: '_vector_37' appears to be a misspelled signal handler [enabled by default]
 ISR(USART1_UDRE_vect)
 ^

On the plus side, the compiled sketch is down from 67,332 to 63,624 bytes, and the reported SRAM usage is down from 3,339 to 3,223 bytes. Just about to test it on the hardware.

[Paolo Paolucci]

Latest nightly on windows XP:

cygwin warning:
  MS-DOS style path detected: G:\arduino-avr-toolchain-nightly-gcc-4.8.1\hardware\arduino\avr\cores\arduino\avr-libc\malloc.c
  Preferred POSIX equivalent is: /arduino-avr-toolchain-nightly-gcc-4.8.1/hardware/arduino/avr/cores/arduino/avr-libc/malloc.c
  CYGWIN environment variable option "nodosfilewarning" turns off this warning. 

ecc ecc..

[Ralph Doncaster]

I'm using W7E-64, but I don't think that's the difference.  It was complaining about:

"...\AppData\Local\Temp\build405672657273688945.tmp/BlinkLED.cpp.elf"
So I think it's the Windows temp variable that is being used for the build dir, and that uses a DOS-style path.

[Federico Fissore]

I've (blindly) added the env var when running things on windows
https://github.com/arduino/Arduino/commit/63a271ebfb93d7a3823c94fa4ddec83e685e5b0e

Current nightly includes both the cygwin fix and the LTO flag

Federico

[Kristian Sloth Lauszus]

Hi,

I just tried to compile the PS3BT example in the USB Host library: https://github.com/felis/USB_Host_Shield_2.0/blob/master/examples/Bluetooth/PS3BT/PS3BT.ino to see the effect of the LTO flag, but to my big supprise the size went up a lot! Here is the result:

Before the LTO flag commit:

Sketch uses 26,628 bytes (82%) of program storage space. Maximum is 32,256 bytes.

Global variables use 984 bytes (48%) of dynamic memory, leaving 1,064 bytes for local variables. Maximum is 2,048 bytes.

After:

Sketch uses 72,984 bytes (226%) of program storage space. Maximum is 32,256 bytes.

Global variables use 1,222 bytes (59%) of dynamic memory, leaving 826 bytes for local variables. Maximum is 2,048 bytes.

Any idea what might be the reason?

Regards

Kristian Lauszus

> an email to developers+unsubscribe@arduino.cc
> <mailto:developers+unsub...@arduino.cc>.

[Mikael Patel]

Below are build statistics for some of the Cosa example sketches (Arduino UNO, program storage space/dynamic memory):

                     1.5.6-r2     4.8.1+lto
Empty Sketch         210/0        154/0
CosaBenchmarkPins    18010/420    13402/451
CosaBlink            1060/21      904/18
CosaBlinkMinimal     266/2        194/0   
CosaCanvasST7735     19790/600    17120/629
CosaSNMP             20936/662    17456/697
CosaMQTThumidity     17528/575    15072/674
CosaThingSpeakClient 23452/989    20884/1046
CosaHD44780          5662/149     4964/196

Cheers!

Mikael

[Paolo Paolucci]

Building empty sketch

In function '__vector_16':
D:\arduino-avr-toolchain-nightly-gcc-4.8.1\hardware\arduino\avr\cores\arduino\wiring.c:47:1: warning: '_vector_16' appears to be a misspelled signal handler [enabled by default]
 ISR(TIMER0_OVF_vect)
 ^

with latest nightly on Win7 64.

> an email to developers+unsubscribe@arduino.cc
> <mailto:developers+unsub...@arduino.cc>.

[Federico Fissore]

Paolo Paolucci, il 29/04/2014 09:49, ha scritto:
> Building empty sketch
>
> In function '__vector_16':
> D:\arduino-avr-toolchain-nightly-gcc-4.8.1\hardware\arduino\avr\cores\arduino\wiring.c:47:1:
> warning: '_vector_16' appears to be a misspelled signal handler
> [enabled by default]
> ISR(TIMER0_OVF_vect)
> ^
>
> with latest nightly on Win7 64.
>

I'm experiencing similar outputs. Attached are the warnings I get
compiling Bridge/YunSerialTerminal example for the Yun

They frankly look scaring and IMHO they are undesirable for the normal
user (non developer/student)

Federico

[Matthijs Kooijman]

Federico,

> > In function '__vector_16':
> > D:\arduino-avr-toolchain-nightly-gcc-4.8.1\hardware\arduino\avr\cores\arduino\wiring.c:47:1:
> > warning: '_vector_16' appears to be a misspelled signal handler [enabled by
> > default]
> > ISR(TIMER0_OVF_vect)
> > ^

As pointed out by Mikael in https://github.com/arduino/Arduino/issues/660,
this warning is caused by a compiler bug and there's no real workaround
(I also checked the gcc sources - you can't disable this warning
specifically). The bug is fixed in 4.8.3:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=59396

Mikael suggested linking with warnings disabled, e.g. add -w to
compiler.c.elf.flags in platform.txt. I'd suggest also adding a comment
with rationale, linking to the above gcc bug report, so we'll remember
to remove the -w flag again when we later upgrade to 4.8.3.

Gr.

Matthijs

[Federico Fissore]

Right. I missed to read that bugzilla.

-w flag added
https://github.com/arduino/Arduino/commit/b119e38581cb900e91c20049f1b30f18e40ed20c

It will be available in the nightly starting tomorrow.

Federico

[Steve Harris]

I've just recompiled my project with the 30th April nightly and get the error:

avr-gcc: error: unrecognized command line option '-flto,--relax'

Steve.

[Steve Harris]

Update:

This error goes away if I change the line in platform.txt:

compiler.c.elf.flags=-Os -Wl,--gc-sections -w -flto

to:

compiler.c.elf.flags=-w -flto -Os -Wl,--gc-sections

I do still get another error though:

/cygdrive/c/Users/steve/AppData/Local/Temp/ccB1vyWD.s: Assembler messages:
/cygdrive/c/Users/steve/AppData/Local/Temp/ccB1vyWD.s:1630: Warning: ignoring changed section attributes for .progmem.data

Steve.

[Federico Fissore]

Thanks Steve, I've pushed the workaround you proposed
https://github.com/arduino/Arduino/commit/f42884daee539e3620bc41b36c469139a966b9eb

and opened an issue on the root cause
https://github.com/arduino/Arduino/issues/2039

The fix will be available starting tomorrow.

I still can't reproduce the "Assembler messages" error you got. Any hint?

Federico

[Steve Harris]

Thanks, Federico.

I'm not sure how to get any more information about the build - are there any additional flags I could set to find out what's causing this?

Steve.

[Federico Fissore]

Steve Harris, il 01/05/2014 10:44, ha scritto:
> Thanks, Federico.
>
> I'm not sure how to get any more information about the build - are there
> any additional flags I could set to find out what's causing this?
>
> Steve.

When I do a test, I use default flags: my HOME/Arduino folder has just a
couple of sketches, boards and platforms.txt are the "vanilla" ones

Did you customized the IDE in someway you can share? Maybe the error is
in one of these additional flags

Federico

[Steve Harris]

It's still happening with the 6th May nightly, but I can compile a blank sketch with no errors.

I'm starting to wonder if it's down to the PROGMEM data I'm using. Could it be that now my compiled Mega2560 sketch is over 64kB that it's starting to move into far pointer territory? There is no single large block of PROGMEM data, but quite a large number of small pieces of data.

Steve.

[Mikael Patel]

I have found a problem with LTO when using a local buffer (on stack) in a member function. A workaround is to use alloca(). See https://github.com/mikaelpatel/Cosa/blob/master/cores/cosa/Cosa/INET/HTTP.cpp#L29

This LTO bug does not show up in all usages of local buffers. I have not investigated the "conditions" in detail and will have to return later with that.

Cheers! Mikael

[Federico Fissore]

Mikael Patel, il 12/05/2014 21:22, ha scritto:
> I have found a problem with LTO when using a local buffer (on stack) in
> a member function. A workaround is to use alloca(). See
> https://github.com/mikaelpatel/Cosa/blob/master/cores/cosa/Cosa/INET/HTTP.cpp#L29
>
> This LTO bug does not show up in all usages of local buffers. I have not
> investigated the "conditions" in detail and will have to return later
> with that.
>
> Cheers! Mikael
>

Hi Mikael

have you find any clues?

Regards

Federico

[Cristian Maglie]

In data lunedì 12 maggio 2014 21:22:31, Mikael Patel ha scritto:
> I have found a problem with LTO when using a local buffer (on stack) in a
> member function. A workaround is to use alloca(). See
> https://github.com/mikaelpatel/Cosa/blob/master/cores/cosa/Cosa/INET/HTTP.c
> pp#L29
>
> This LTO bug does not show up in all usages of local buffers. I have not
> investigated the "conditions" in detail and will have to return later with
> that.

Hi Mikael,

while reading I thought that we forget about the malloc patch in

hardware/arduino/avr/cores/arduino/avr-libc/*

were you using a similar patch for COSA? If yes, may it be the cause of the
issue?

The patch must be removed from the Arduino core since the new toolchain
provides an updated malloc implementation.

C

[Cristian Maglie]

In data lunedì 28 aprile 2014 23:12:17, Kristian Sloth Lauszus ha scritto:
> Hi,
>
> I just tried to compile the PS3BT example in the USB Host library:
> https://github.com/felis/USB_Host_Shield_2.0/blob/master/examples/Bluetooth

> /PS3BT/PS3BT.ino to see the effect of the LTO flag, but to my big supprise

> the size went up a lot! Here is the result:
>

Hey Kristian,

have you find the reason for this issue?

It seems that there are a bunch of concerns around LTO, if we can't sort all
them out we should disable it.

C

[Mikael Patel]

Hi Cristian!

No - sorry I have not done any more digging into the problem with LTO I
had with the HTTP server support class.

Have not looked at the old malloc patch mainly because Cosa is built
with a "no-malloc/free (new/delete)" strategy.

As Cosa is over 50 KLOC commented source and 140+ example/test sketches
and it was so far has only been this problem with stack allocated buffer
I believe the LTO to be quite stable. Though more testing is needed.
Also I would like to check the assemble listing for the function that
failed.

My focus has been adding a layer of support for multiple vendors and
boards and adding more documentation (circuit/pin-out descriptions). And
adding support for Teensy++ 2.0 and a SPI device driver for cascaded
shift-registers (74HC595).

Hope to get some time over for the LTO testing again.

Cheers!

Mikael

[Kristian Sloth Lauszus]

Hi Christian,

No I haven't been able to fix the issue, but I haven't look into it that much as I got exams right now.

The only thing I could find was this link: http://avr.2057.n7.nabble.com/binutils-2-22-blow-up-program-size-with-flto-td5405.html. The issue sounds very similar to the one I got.

Regards

Lauszus

[Cristian Maglie]

Hi Mikael,

In data lunedì 19 maggio 2014 23:51:59, Mikael Patel ha scritto:
> Have not looked at the old malloc patch mainly because Cosa is built
> with a "no-malloc/free (new/delete)" strategy.

Impressive! May I ask you how you handle strings or variable length content
(tipically network streams)?

On 05/19/2014 03:41 PM, Cristian Maglie wrote:
> while reading I thought that we forget about the malloc patch in
>
> hardware/arduino/avr/cores/arduino/avr-libc/*

The malloc patch has been removed on the new toolchain nightly:

https://github.com/arduino/Arduino/commit/ec283825bc734b6711bc733657d97c787c46d1d4

C

[Mikael Patel]

On 05/22/2014 10:46 PM, Cristian Maglie wrote:
> Hi Mikael,
>
> In data lunedì 19 maggio 2014 23:51:59, Mikael Patel ha scritto:
>> Have not looked at the old malloc patch mainly because Cosa is built
>> with a "no-malloc/free (new/delete)" strategy.
> Impressive! May I ask you how you handle strings or variable length content
> (tipically network streams)?

That is a very good question. First I try to use stack based allocation
instead but also as much of the memory available in devices etc. For
instance, the Cosa Ethernet (W5100) device driver makes use of the
buffers in the device and maps that to an IOStream. This allows for
instance a single web-server sketch to use less than 100 bytes of stack
to create all strings etc.
https://github.com/mikaelpatel/Cosa/blob/master/examples/Ethernet/CosaWebServer/CosaWebServer.ino#L56
The magic is in line 79 where the socket is bound to an IOStream and all
the need for creating strings and concatenating "vanishes". Strings are
streamed directly to the device buffer and transmitted when full or on
demand.

The Cosa IOBuffer may be used for creating temporary strings (much like
a max size bound String). The same goes for Offscreen Canvas which is
also fixed memory and often used "on stack".
https://github.com/mikaelpatel/Cosa/blob/master/examples/LCD/CosaPCD8544temp/CosaPCD8544temp.ino#L98
Instead of a global buffer for the PCD8544 bitmap an Offscreen Canvas is
created on stack.

This doesn't really answer your question directly. It is more a strategy
of development where the stack is used instead (combined with binary
formats). The overhead of malloc/free together with heap fragmentation
and risk of giving the illusion that there is more memory then there is
are the main arguments against using heap on small scale embedded. With
larger memory size (> 4K) and larger applications a heap becomes
necessary. But then the head room is also larger.

Cheers! Mikael

[Mikael Patel]

Some measurements that might be of interest. I have run the Cosa Pins Benchmark in Arduino 1.0.5, 1.5.6-r2 and the new tool chain (GCC 4.8.1) with and without link time optimization (-flto).

The results may be found here; https://dl.dropboxusercontent.com/u/993383/Cosa/calc/CosaBenchmarkPins.ods
The benchmark code is here: https://github.com/mikaelpatel/Cosa/blob/master/examples/Benchmarks/CosaBenchmarkPins/CosaBenchmarkPins.ino

Please note that the benchmark is only pin access function (equivalent to Arduino/Wiring digitalRead/Write, etc). All measurements are in ns. The raw data is in https://github.com/mikaelpatel/Cosa/tree/master/examples/Benchmarks/CosaBenchmarkPins.

Cheers! Mikael

[Federico Fissore]

Thank you Mikael, very nice work! It seems like there is an average
improvement of 1.4, if you don't consider the spikes in input reading
and in digitalWrite to calculate it. And that's good news

I'm puzzled by the difference between
OutputPin::write(D8, val)
and
digitalWrite(D8, val)

If I understand it correctly they should do the same thing and the
difference should be just this "if"
https://github.com/mikaelpatel/Cosa/blob/master/examples/Benchmarks/CosaBenchmarkPins/CosaBenchmarkPins.ino#L67
Can it alone make digitalWrite ten times slower?

Federico

[Mikael Patel]

Den fredagen den 30:e maj 2014 kl. 09:18:34 UTC+2 skrev f.fissore:

I'm puzzled by the difference between
OutputPin::write(D8, val)
and
digitalWrite(D8, val)

If I understand it correctly they should do the same thing and the
difference should be just this "if"
https://github.com/mikaelpatel/Cosa/blob/master/examples/Benchmarks/CosaBenchmarkPins/CosaBenchmarkPins.ino#L67
Can it alone make digitalWrite ten times slower?

Hi! The logic of the optimizer is not always easy to understand as there are a number of control parameters involved. GCC 4.8.1 generates much better code but does not inline as much as previous version (in -Os). The test you point to should resolve into one of the "write" functions as the parameters are both constant at compile time. My guess (without looking at the assemble code) is that optimizer does not inline and continue the elimination. This might be due to the call depth. I need to do a deeper analysis before coming up with a better explanation. BW: The "if" should not generate any code at all. 

The link-time optimization (-flto) step does give some additional improvements but I am concerned with some effects that it has such as increased code size and possible error with stack buffers (as I found previously). However I would recommend getting 4.8.1 support into the next release of 1.5.X. The gain is a quantum leap forward.

Cheers! Mikael

[Mikael Patel]

I have updated the benchmark with more information (board, memory size, etc). Also made the constant parameter optimization optional for the Arduino IDE 1.0.X. This removes the spike and give a more "fair" comparison. Links as above.

Cheers! Mikael

[Federico Fissore]

Amazing work, thank you! 1.91 times faster, cool :)

[Kristian Sloth Lauszus]

Christian I can see that you have merged in the lto flag changes to platform.txt into the ide-1.5.x branch: https://github.com/arduino/Arduino/commit/370648a53298cabb2e8a17ee28e11e5ef2edb691#diff-33, but I think that it is a mistake as it blows up the code size in the USB Host library, as I have written before.

Here is the output when compiling using 1.5.6-r2 for an Arduino ADK:

Sketch uses 13,942 bytes (5%) of program storage space. Maximum is 258,048 bytes.

Global variables use 565 bytes (6%) of dynamic memory, leaving 7,627 bytes for local variables. Maximum is 8,192 bytes.

And here is the output when using the newest version on Github:

Sketch uses 73,730 bytes (28%) of program storage space. Maximum is 258,048 bytes.

Global variables use 734 bytes (8%) of dynamic memory, leaving 7,458 bytes for local variables. Maximum is 8,192 bytes.

As you can see it makes the library unusable for most users! So please consider removing the flag again until we have come up with a fix. 

Regards

Lauszus

[Cristian Maglie]

In data domenica 29 giugno 2014 14:02:10, Kristian Sloth Lauszus ha scritto:
> As you can see it makes the library unusable for most users! So please
> consider removing the flag again until we have come up with a fix.

Hey,

this is indeed a show-stopper for LTO.
Until we understand what's going on, I'm going to disable it for the moment.

C

[Ralph Doncaster]

I decided to look into this, and although I haven't figured it out, there's some things I tried that can be ruled out as a source of the problem:

- removing -ffunction-sections -fdata-sections 
- removing -g

It doesn't seem to be an inlining issue, as -fno-inline didn't help.  I tried -O2 & -O3, and both made the binary even larger.

Looking at the disassembled binary, the problem seems to be unused functions and text are not getting removed.  I was building the USBHIDJoystick example, and here's a couple examples of code and data that I think is unused but still gets included in the binary:

0000017d <_ZL26pstrUsageSystemDisplayDual.5844.2717>:

0000807e <_ZN7Max_LCD5writeEh.local.39.1865>:

[Ralph Doncaster]

I figured out why a lot of unused code is included in the build.

Rather than try to figure out the arduino build process I tried building from the command line.  I did it with MinGW avr-gcc 4.8.0 since it was quicker than setting up the paths to avr-gcc contained in the arduino-nightly package.

As with Lauszus' results, the first successful build from the command line was bloated:

$ avr-gcc -mmcu=atmega328p -Os -flto core.a USBHIDJoystick.cpp.o hidjoystickrptparser.cpp.o USB_Host_Shield_2.0-master/*.o -o USBHIDJ-full.elf

$ avr-size USBHIDJ-full.elf

   text    data     bss     dec     hex filename

  83834     838     469   85141   14c95 USBHIDJ-full.elf

I previously noticed the linker would only exclude unused functions when they are included in an archive.  Running avr-gcc foo.o bar.o -o main.elf will include the code from bar.o even when bar.o (containing main) does not reference anything in bar.o.

So I created USB_Host_Shield_2.0-master/UHS.a, linked with the archive instead, and got much better results:

$ avr-gcc -mmcu=atmega328p -Os -flto core.a USBHIDJoystick.cpp.o hidjoystickrptparser.cpp.o USB_Host_Shield_2.0-master/UHS.a -o USBHIDJ.elf

$ avr-size USBHIDJ.elf

   text    data     bss     dec     hex filename

  15912     212     465   16589    40cd USBHIDJ.elf

Adding -Wl,relax saves a few more bytes:

$ avr-size USBHIDJ.elf

   text    data     bss     dec     hex filename

  15522     212     465   16199    3f47 USBHIDJ.elf

[Federico Fissore]

Very nice analysis, it looks like gcc is acting weird

Ralph Doncaster, il 02/07/2014 03:13, ha scritto:
> So I created USB_Host_Shield_2.0-master/UHS.a, linked with the archive
> instead, and got much better results:
> $ avr-gcc -mmcu=atmega328p -Os -flto core.a USBHIDJoystick.cpp.o
> hidjoystickrptparser.cpp.o USB_Host_Shield_2.0-master/UHS.a -o USBHIDJ.elf
>

You mean you made an archive with only one object file?

Federico

[Matthijs Kooijman]

Hey Ralph,

> I figured out why a lot of unused code is included in the build.

Ah, was it established that the bigger code size was because of unused
code? I had assumed it was due to less efficient optimization, but this
is easier to debug I think :-)

> I previously noticed the linker would only exclude unused functions when
> they are included in an archive. Running avr-gcc foo.o bar.o -o main.elf
> will include the code from bar.o even when bar.o (containing main) does not
> reference anything in bar.o.

Your probably meant "foo.o (containing main)" there?

As for archives - indeed, the linker only takes compilation units from
an archive that it actually needs, unlike .o files directly specified
which are unconditionally included in the build.

However, after selecting the compilation units to include in the build
the gc-sections option combined with -ffunction-sections and
-fdatasections should make sure that any functions or variables that end
up not being used, should be removed from the build again. So perhaps
lto somehow prevents that mechanism from working properly?

Gr.

Matthijs

[Ralph Doncaster]

No, I created USB_Host_Shield_2.0-master/UHS.a containing all the .o files that were in the subdir.  i.e.:

avr-ar rcs USB_Host_Shield_2.0-master/UHS.a USB_Host_Shield_2.0-master/*.o

 

[Ralph Doncaster]

On Wednesday, 2 July 2014 06:11:09 UTC-3, Matthijs Kooijman wrote:

Hey Ralph,

> I figured out why a lot of unused code is included in the build.
Ah, was it established that the bigger code size was because of unused
code? I had assumed it was due to less efficient optimization, but this
is easier to debug I think :-)

Yes, it's just due to including code from .o files specified on the command line but that are not actually used.  When using an archive only the required .o's are extracted and linked (and hopefully only the required symbols from those .o's and not the whole .o)

 

> I previously noticed the linker would only exclude unused functions when
> they are included in an archive.  Running avr-gcc foo.o bar.o -o main.elf
> will include the code from bar.o even when bar.o (containing main) does not
> reference anything in bar.o.
Your probably meant "foo.o (containing main)" there?

Ahh, yes.  My Canada Day celebrations yesterday involved consuming 1.5L of beer, so that may have influenced my accuracy. :-)

 

As for archives - indeed, the linker only takes compilation units from
an archive that it actually needs, unlike .o files directly specified
which are unconditionally included in the build.

However, after selecting the compilation units to include in the build
the gc-sections option combined with -ffunction-sections and
-fdatasections should make sure that any functions or variables that end
up not being used, should be removed from the build again. So perhaps
lto somehow prevents that mechanism from working properly?  

I agree.  I suspect (but haven't gone through the gcc source to confirm) that lto and -f.*-sections are mutually exclusive, since lto does everything -f.*-sections does and more.

Since the norm in C/C++ is to build libraries into archives and then link to the library archives, it would explain why this oddity with LTO has gone unnoticed.

I would think the way Arduino includes all the .o's should have caused problems earlier - like for older versions of windows with rather short command line length limits.

-Ralph

[Matthijs Kooijman]

Hey Ralph,

> When using an archive only the required .o's are extracted and linked
> (and hopefully only the required symbols from those .o's and not the
> whole .o)

I think the entire .o files are used - not just the required symbols
(this is why C libraries often have one function per .c file). We
actually use this in the Arduino core - when including
HardwareSerial1.cpp, you also get the corresponding ISR in the link even
though it's not referenced.

> > As for archives - indeed, the linker only takes compilation units from
> > an archive that it actually needs, unlike .o files directly specified
> > which are unconditionally included in the build.
> >
> > However, after selecting the compilation units to include in the build
> > the gc-sections option combined with -ffunction-sections and
> > -fdatasections should make sure that any functions or variables that end
> > up not being used, should be removed from the build again. So perhaps
> > lto somehow prevents that mechanism from working properly?
> >
>
> I agree. I suspect (but haven't gone through the gcc source to confirm)
> that lto and -f.*-sections are mutually exclusive, since lto does
> everything -f.*-sections does and more.

Well, apparently it doesn't ;-)

But indeed, it sounds like -flto should be able to throw away unused
functions by itself already, provided that it is really seeing the
entire program. Two things that might be worth checking:
- Is there really a linker plugin active, that allows looking inside .a
files for lto info and symbol references?
- Do all .o and .a files actuall contain LTO bytecode?

A somewhat unrelated note, I just read this in the gcc 4.9.0 manpage:

Link-time optimization does not work well with generation of
debugging information. Combining -flto with -g is currently
experimental and expected to produce wrong results.

So if we add -flto, we should probably remove -g, even though it doesn't
seem to be the cause of our current problem (as tested by you).

Gr.

Matthijs

[Ralph Doncaster]

On Wednesday, 2 July 2014 14:51:49 UTC-3, Matthijs Kooijman wrote:

Hey Ralph,

> When using an archive only the required .o's are extracted and linked
> (and hopefully only the required symbols from those .o's and not the
> whole .o)
I think the entire .o files are used - not just the required symbols
(this is why C libraries often have one function per .c file). We
actually use this in the Arduino core - when including
HardwareSerial1.cpp, you also get the corresponding ISR in the link even
though it's not referenced.

I tested this out and you are correct.

 

> > As for archives - indeed, the linker only takes compilation units from
> > an archive that it actually needs, unlike .o files directly specified
> > which are unconditionally included in the build.
> >
> > However, after selecting the compilation units to include in the build
> > the gc-sections option combined with -ffunction-sections and
> > -fdatasections should make sure that any functions or variables that end
> > up not being used, should be removed from the build again. So perhaps
> > lto somehow prevents that mechanism from working properly?  
> >
>
> I agree.  I suspect (but haven't gone through the gcc source to confirm)
> that lto and -f.*-sections are mutually exclusive, since lto does
> everything -f.*-sections does and more.
Well, apparently it doesn't ;-)

But indeed, it sounds like -flto should be able to throw away unused
functions by itself already, provided that it is really seeing the
entire program. Two things that might be worth checking:
 - Is there really a linker plugin active, that allows looking inside .a
   files for lto info and symbol references?
 - Do all .o and .a files actuall contain LTO bytecode?

 

Good questions.  I was't as systematic as I could've been, so maybe the .o's in the USB_Host_Shield_2.0-master subdir didn't have lto symbols.  I did some more testing and the behavior is not dependant on the lto flag.

I make linktest.o which references foo (contained in foobar.a).  I linked with crc16usb.o, and it was included in the binary even though none of it was used.  -Wl,-relax and -fwhole-program made no difference.

 $ avr-gcc -mmcu=attiny84 -Os linktest.o crc16usb.o foobar.a

With crc16usb.a, the linker didn't pull it in.

I'm pretty sure there's an active linker plugin with the arduino-nightly build, because adding -fwhole-program to the build didn't make an difference.  However with the AVR toolchain 3.4.4, I've documented how it doesn't have the linker plugin (and reported it to Atmel support).

http://nerdralph.blogspot.ca/2014/06/atmel-avr-toolchain-344.html

Checking the dumpspecs output from the arduino nightly build would confirm whether or not the linker plugin is included.

Running objdump on the USB_Host_Shield_2.0-master .o files would confirm whether lto symbols are include.

A somewhat unrelated note, I just read this in the gcc 4.9.0 manpage:

        Link-time optimization does not work well with generation of
        debugging information.  Combining -flto with -g is currently
        experimental and expected to produce wrong results.

So if we add -flto, we should probably remove -g, even though it doesn't
seem to be the cause of our current problem (as tested by you).

I agree.  As a side effect the build should go a little faster without debug symbols included.

-Ralph

 

[Matthijs Kooijman]

Hey Ralph,

> Good questions. I was't as systematic as I could've been, so maybe the
> .o's in the USB_Host_Shield_2.0-master subdir didn't have lto symbols. I
> did some more testing and the behavior is not dependant on the lto flag.
> I make linktest.o which references foo (contained in foobar.a). I linked
> with crc16usb.o, and it was included in the binary even though none of it
> was used. -Wl,-relax and -fwhole-program made no difference.
> $ avr-gcc -mmcu=attiny84 -Os linktest.o crc16usb.o foobar.a
>
> With crc16usb.a, the linker didn't pull it in.

You didn't specify the gc-sections option, so it's expected that any .o
files directly on the commandline are included in their entirety.

> I'm pretty sure there's an active linker plugin with the arduino-nightly
> build, because adding -fwhole-program to the build didn't make an
> difference. However with the AVR toolchain 3.4.4, I've documented how it
> doesn't have the linker plugin (and reported it to Atmel support).
> http://nerdralph.blogspot.ca/2014/06/atmel-avr-toolchain-344.html

Right, I believe so as well.

Gr.

Matthijs

[Ralph Doncaster]

On Wednesday, 2 July 2014 14:51:49 UTC-3, Matthijs Kooijman wrote:

 - Do all .o and .a files actuall contain LTO bytecode?

Just finished checking and it looks like they all do, or at least they contain the symbol __gnu_lto_v1 (based on egrep -l __gnu_lto_).

I've done some more fiddling around and discovered that order of arguments makes a difference. If I don't include core.a first, I get a bunch of link errors.

$ avr-gcc -mmcu=atmega328p -flto -Wl,-relax -Os USB_Host_Shield_2.0-master/UHS.

a core.a hjrp.a -o USBHIDJ.elf

`millis' referenced in section `.text' of USB_Host_Shield_2.0-master/UHS.a(Usb.c

pp.o): defined in discarded section `.text' of wiring.c.o (symbol from plugin)

[...]

This lead me to investigate how the callgraph for dead code is generated with LTO.  Each .o will be used as a starting point for a callgraph root, so only main.o should be listed on the link line -- everything else should be in archives.  So here's what the Arduino IDE needs to do to get the best out of LTO:

avr-gcc -mmcu=atmega328p -Wl,-relax -flto -Os main.cpp.o hjrp.a core.a USB_Host_Shield_2.0-master/UHS.a -o USBHIDJ.elf

That gets the binary down to 12530 bytes - even smaller than my previous results.

I've written a blog post with the full details here:

http://nerdralph.blogspot.ca/2014/07/gcc-lto-call-graph-generation.html

[Matthew Ford]

gcc-4.9.0 seems to be terminally broken
http://lkml.iu.edu//hypermail/linux/kernel/1407.3/00650.html

Hopefully Atmel and AVR (and Arduino) will skip this version of GCC

matthew

--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[William "Chops" Westfield]

On Jul 27, 2014, at 3:34 PM, Matthew Ford wrote:

> gcc-4.9.0 seems to be terminally broken
> http://lkml.iu.edu//hypermail/linux/kernel/1407.3/00650.html

In an area that is inapplicable to AVR ?

In any case, if arduino is going to track new compiler releases, it should probably track the versions released by Atmel, rather than the un-vetted gcc releases. It's not at all uncommon for a "working fine" gcc version to have serious bugs on "unusual" targets like the AVR…

WestfW

[Ralph Doncaster]

Following some suggestions from Jan Hubička  I tried adding -fuse-linker-plugin, but that didn't help.  Nor does -fwhole-program.

I'm also confused over whether avr-ld has linker plugin support.

avr-ld --help shows -plugin as one of the options, however running avr-ld -plugin says "unrecognized option '-plugin'"

I think binutils should be updated from 2.23.2 to 2.24.  The latest atmel toolchain for linux (and presumably the windows version too) includes binutils 2.24 along with gcc 4.8.1:

# avr-gcc --version

avr-gcc (AVR_8_bit_GNU_Toolchain_3.4.4_1229) 4.8.1

Copyright (C) 2013 Free Software Foundation, Inc.

# avr-ld --version

GNU ld (AVR_8_bit_GNU_Toolchain_3.4.4_1229) 2.24

Copyright 2013 Free Software Foundation, Inc.