FYI: Performance improvement for Print::print(const String &s)

[Federico Fissore]

Hi everyone,

after a user noticed the poor performance of

Print::print(const String &s)

compared to that of

Print::write(const uint8_t *buffer, size_t size),

we have fixed it with the following code

size_t Print::print(const String &s)
{
return write(reinterpret_cast<const uint8_t*>(s.c_str()), s.length());
}

Additional details can be found at
https://github.com/arduino/Arduino/issues/1760

Please provide any feedback in case this impacts negatively any of your
projects

Thank you in advance

Federico

[Rob Tillaart]

hit the wrong reply

---------- Forwarded message ----------
From: Rob Tillaart <rob.ti...@gmail.com>
Date: Mon, Dec 23, 2013 at 8:05 PM
Subject: Re: [Developers] FYI: Performance improvement for Print::print(const String &s)
To: Federico Fissore <f.fi...@arduino.cc>

This seems a good improvement. 

Can you explain why you used a reinterpret_cast?

a const_cast works perfectly and is less "insecure" - http://www.cplusplus.com/doc/tutorial/typecasting/ -

size_t Print::print(const String &s)
{

 return write(const_cast<const uint8_t*>(s.c_str()), s.length());
}

Federico

--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@arduino.cc.

[Cristian Maglie]

In data lunedì 23 dicembre 2013 20:12:22, Rob Tillaart ha scritto:
> This seems a good improvement.
> Can you explain why you used a reinterpret_cast?
> a const_cast works perfectly and is less "insecure" -
> http://www.cplusplus.com/doc/tutorial/typecasting/ -
>
>
> size_t Print::print(const String &s)
> {
> return write(const_cast<const uint8_t*>(s.c_str()), s.length());
> }

this won't work because c_str() returns a

const char *

while we need a

const uint8_t *

we are not setting "constness" but changing the pointer type.

C

[Rob Tillaart]

@Federico

There are more improvements for the Print class, please check if one of these can be included.

1)  improve print performance for all numbers with divmod10 code (avr)   - no issue - 

(A) - http://forum.arduino.cc/index.php?topic=167414.0 - divmod10 code

(B) - http://forum.arduino.cc/index.php?topic=179111.0 - divmod10 applied in the print function.

Together with Darryl, PaulS and others, the performance of print increased substantially,

and as print is a base class for Serial, ethernet etc, the gain will be everywhere.

2) printing HEX numbers always with leading zero - issue 1097

Darryl posted code to print with leading zero's, an item often mentioned on the forum.

see (B) above

3) Support for SCI and ENG and SI format - related to  issue #1412

Useful for floats especially if bigger than maxlong.

- http://forum.arduino.cc/index.php?topic=179111.msg1375203#msg1375203 - 

4) Support for long long printing - issue #1236

- http://forum.arduino.cc/index.php?topic=143584.msg1078250#msg1078250 - 

as proposed it is conditional compiled as 64 bit takes quite some space.

5) (small) memory improvement of the print.cpp lib - issue #1222 

-----

might need more than one release to include all of these ;)

On Mon, Dec 23, 2013 at 11:58 AM, Federico Fissore <f.fi...@arduino.cc> wrote:

Federico

[Rob Tillaart]

A quick test compiled without warning ...?

C

--
You received this message because you are subscribed to the Google Groups "Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[Cristian Maglie]

In data lunedì 23 dicembre 2013 20:59:03, Rob Tillaart ha scritto:
> A quick test compiled without warning ...?

I have:

Print.cpp: In member function ‘size_t Print::print(const String&)’:
Print.cpp:56: error: invalid const_cast from type ‘const char*’ to type ‘const
uint8_t*’


C

[Rob Tillaart]

Oops my mistake, changed the wrong print.cpp (have 3 different configs side by side)

C

[Matthijs Kooijman]

Hey Christian,

> this won't work because c_str() returns a
>
> const char *
>
> while we need a
>
> const uint8_t *

Wouldn't a better solution be to add a Print::write(const char*, size_t)
version as well (which just does the cast and calls the existing write
function). This would also make things easier for novice users - I've
seen beginners struggle with char* vs uint8_t* (especially since
Stream::readBytes only works with char*).

Since suggestions are always best expressed and motivated with code,
here's the pullreq:

https://github.com/arduino/Arduino/pull/1762

It adds a char* version of Print::write and uint8_t* versions of all
multiple byte reading methods in Stream (which simply cast and then call
the existing method). This should not have any code size overhead
because the new methods will be inlined. All existing sketches will also
still work since this just adds to the API, without changing any of the
existing API.

Gr.

Matthijs

[Matthijs Kooijman]

Hey Rob,

> There are more improvements for the Print class, please check if one of
> these can be included.
>
> 1) improve print performance for all numbers with divmod10 code (avr) -
> no issue -
>
> (A) - http://forum.arduino.cc/index.php?topic=167414.0 - divmod10 code
> (B) - http://forum.arduino.cc/index.php?topic=179111.0 - divmod10 applied
> in the print function.
>
> Together with Darryl, PaulS and others, the performance of print increased
> substantially,
> and as print is a base class for Serial, ethernet etc, the gain will be
> everywhere.
>
> 2) printing HEX numbers always with leading zero - issue 1097
> Darryl posted code to print with leading zero's, an item often mentioned on
> the forum.
> see (B) above
>
>
> 3) Support for SCI and ENG and SI format - related to issue #1412
> Useful for floats especially if bigger than maxlong.
> - http://forum.arduino.cc/index.php?topic=179111.msg1375203#msg1375203 -
>
>
> 4) Support for long long printing - issue #1236
> - http://forum.arduino.cc/index.php?topic=143584.msg1078250#msg1078250 -
> as proposed it is conditional compiled as 64 bit takes quite some space.
>
> 5) (small) memory improvement of the print.cpp lib - issue #1222

I had a a quick look at some of the code, that looks good. Perhaps
someone should turn that code into a pullrequest (against the ide-1.5.x
branch would be best I think)?

To make it easy for review, I'd suggest separating everything into
different commits (e.g. a separate commit per point above).

Gr.

Matthijs

[Rob Tillaart]

Hi Matthijs,

(pull request) true, should be done, time is a choice ...

1) the SCI /ENG/ SI notation is imho the most important one. It allows to print floats in their whole range and use a "fixed" length on a display.

2) the divmod10 should be in Arduino.h or better in a fastMath.h so that every lib/class can use the routines. 

There are more fast math routines on the forum, e.g divmod60,divmod24 for seconds, minutes hours and days.

in a 2nd step it can be included in the print class.

Thanks for your feedback,

Rob

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAlK5i2EACgkQz0nQ5oovr7xcYQCgs9GbYLr1xXLnzTean35DWGWB
5r4AnjnvyMmlBQouBibBIafWG/gpIhit
=z5IE
-----END PGP SIGNATURE-----

[Cristian Maglie]

Hi Rob,

I've been looking at all your suggested improvements:

In data lunedì 23 dicembre 2013 20:57:54, Rob Tillaart ha scritto:

> 1) improve print performance for all numbers with divmod10 code (avr) -

> no issue -

>

> (A) - http://forum.arduino.cc/index.php?topic=167414.0 - divmod10 code

> (B) - http://forum.arduino.cc/index.php?topic=179111.0 - divmod10 applied

> in the print function.

>

> Together with Darryl, PaulS and others, the performance of print increased

> substantially,

> and as print is a base class for Serial, ethernet etc, the gain will be

> everywhere.

This is a very nice improvement. As you already suggested, we may start by providing a "fast_math.h" that contains all the optimized code for generic math calculations. The fast_math will not be intended for normal usage in user's sketches, but only for core/libary developers to optimize critical parts of code. Moreover we must provide a similar file for every architecture (at least to provide something that fall-back to standard math subroutines).

> 2) printing HEX numbers always with leading zero - issue 1097

> Darryl posted code to print with leading zero's, an item often mentioned on

> the forum.

> see (B) above

AFAIU this patch changes the print(__,HEX) behaviour, breaking existing sketches that relies on printing without leading zeros.

A different solution may be something like the following:

Serial.print(x, HEX, 4) where 4 is the number of digits to print (including leading zeros), as described here:

https://github.com/arduino/Arduino/issues/549

http://code.google.com/p/arduino/issues/detail?id=549

> 3) Support for SCI and ENG and SI format - related to issue #1412

> Useful for floats especially if bigger than maxlong.

> - http://forum.arduino.cc/index.php?topic=179111.msg1375203#msg1375203 -

I'm not sure I fully understand the difference between the three formats, may you explain it with some examples? This will help to understand the use cases behind this change.

I see also that you handle directly bits in the binary floating point layout (to grab the exponential part from the 32-bit floating point raw data). I know that AVR doesn't support "double", while ARM does, maybe this need to be handled in some way?

I'm wondering if these subroutines deserves a specific library (with other math functions?) instead of trying to push them in the core.

> 4) Support for long long printing - issue #1236

> - http://forum.arduino.cc/index.php?topic=143584.msg1078250#msg1078250 -

> as proposed it is conditional compiled as 64 bit takes quite some space.

Probably we can just add the methods to print int64 since the extra space should be taken only when used.

> 5) (small) memory improvement of the print.cpp lib - issue #1222

I see a bunch of them in the forum thread, I'll try to pick the simplest ones to begin.

C

[Rob Tillaart]

> The fast_math will not be intended for normal usage in user's sketches, but only for core/libary developers to optimize critical parts of code.

People should be able to use it, but they should understand what they working with. They are also allowed to use the twi lib themselves but most people just don't do that. But it would be great to have those optimized routines separated from the applications. 

----

2) with respect to the HEX printing:

. 

we can also implement - char * hex(number); which will be optimized for converting numbers to a hex representation .

1 byte types would print 2 digits  NN - UPPER case

2 byte types would print 4 digits  NNNN

4 byte types would print 8 digits  NNNNNNNN

etc

hex(ptr, len) would do a nice hex dump of len bytes with spaces:  NN NN NN NN ... NN 

hex(float) is a question

I could do :  

int dozen = 12;

int five = 5;

Serial.println( hex(dozen) );  // prints 0C 

Serial.println( hex(five) );  // prints 05 

SImilar an optimized char * bin() that always has a multiple of 8 digits.

This way we do not need to patch the print() code, and wont break interfaces. 

> 3) Support for SCI and ENG and SI format - related to issue #1412

> Useful for floats especially if bigger than maxlong.

> - http://forum.arduino.cc/index.php?topic=179111.msg1375203#msg1375203 -

I'm not sure I fully understand the difference between the three formats, may you explain it with some examples? This will help to understand the use cases behind this change.

SCI = SCIentific.  ENG = ENgineering.  SI = SI units.

SCI format  = n.nnnnnEnn  - 1 digit for the point;  several after the point (param)   followed by an E and the exponent. 

ENG format is similar to SCI, but the exponent is always a multiple of 3.  This multiple of 3 matches the prefixes - milli  micro  Kilo mega etc.

This multiple of 3 also implies that the number of digits before the point can be 1,2 or 3.

SCI and ENG can be found on calculators.

SI is no official format, it is made on request on the forum. It replaces the Exponent with the equivalent SI character for milli micro Kilo etc.

4) agree, I had put conditional #ifdef around it but that is not necessary (stupid me :)

> 5) (small) memory improvement of the print.cpp lib - issue #1222

I see a bunch of them in the forum thread, I'll try to pick the simplest ones to begin.

Agree, only take those that are clear in how they affect the code (e.g. base)

If there are more things that need clarification, let me know. 

Regards,

Rob

--
You received this message because you are subscribed to the Google Groups "Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[Rob Tillaart]

a straightforward implementation of the hex() and bin() conversion with leading zero's. (no separating spaces)

char * hex(uint32_t nr, uint8_t digits)

{

  static char buf[10];

  buf[digits] = '\0';

  digits--;

  for (int i=digits; i>=0; i--)

  {

    uint8_t n = nr & 0x0F;

    if (n <10) buf[i] = '0' + n;

    else buf[i] = 'A' - 10 + n;

    nr >>= 4;

  }

  return buf;

}

char * bin(uint32_t nr, uint8_t digits)

{

  static char buf[33];

  buf[digits] = '\0';

  digits--;

  for (int i=digits; i>=0; i--)

  {

    buf[i] = '0' + (nr & 1);

    nr >>= 1;

  }

  return buf;

}

[Matthijs Kooijman]

Hey Rob,

> > The fast_math will not be intended for normal usage in user's sketches,
> > but only for core/libary developers to optimize critical parts of code.
> People should be able to use it, but they should understand what they
> working with. They are also allowed to use the twi lib themselves but most
> people just don't do that. But it would be great to have those optimized
> routines separated from the applications.

I agree, it would be good to offer these for advanced users as well.

> 2) with respect to the HEX printing:
> .
> we can also implement - char * hex(number); which will be optimized for
> converting numbers to a hex representation .
>
> 1 byte types would print 2 digits NN - UPPER case
> 2 byte types would print 4 digits NNNN
> 4 byte types would print 8 digits NNNNNNNN
> etc

I agree that something that has a default number of digits based on the
actual integer type would be very useful.

> Serial.println( hex(dozen) ); // prints 0C
> Serial.println( hex(five) ); // prints 05

This looks like acceptable syntax to me.

However, I do think that implementing your proposal is hard: where do
you allocate the memory for the string? If you have a single static buffer,
then you can get reentrancy problems. e.g., you can't do:

one = hex(1);
two = hex(2);

Additionally, you'll have to decide at compile time what the maximum
size of the string can be and then always use that many bytes, even if
you're not using them all.

If you do dynamic memory, then there is no automatic way to free the
memory again (unless you do autoptr style stuff, but that isn't used at
all in Arduino yet). E.g., you have to do:

one = hex(1);
Serial.println(one);
free(one);

Which defeats the point of the concise syntax.


Something we could perhaps do would be to have a hex() function that
simply returns the original number, hidden in a struct type or something
like that, with overloaded print functions to handle it (e.g., a bit
like the __FlashStringHelper type that's already used)?

Gr.

Matthijs

[Rob Tillaart]

Thanks Matthijs for this feedback,

> However, I do think that implementing your proposal is hard: where do
> you allocate the memory for the string? If you have a single static buffer,
> then you can get reentrancy problems. e.g., you can't do:
>
>        one = hex(1);
>        two = hex(2);

You have a serious point here!

 I can imagine how many discussions that would give on the forum :)

So as the HEX() function is 99.9% for printing (assumption), we can add the following methods to the print Class 

// not tested code

size_t printHEX(uint8_t nr)

{

  return printHEX((uint16_t) nr, 2);  // forces 2 digits

}

size_t printHEX(int8_t nr)

{

  return printHEX((uint16_t) nr, 2);  // forces 2 digits

}

size_t printHEX(uint16_t nr)

{

  return printHEX((uint16_t) nr, 4);   // forces 4 digits

}

size_t printHEX(int16_t nr)

{

  return printHEX((uint16_t) nr, 4);   // forces 4 digits

}

size_t printHEX(uint32_t nr)   // NNNNNNNN

{

  size_t n = printHEX((uint16_t) nr >> 8,  4);

  n += printHEX((uint16_t) nr & 0xFFFF, 4);

  return n;

}

size_t printHEX(int32_t nr)

{

  return  printHEX((uint32_t) nr);

}

// the workhorse is 16 bit as it is faster than 32 bit, and I expect most 'hexdumps' to be either 8 or 16 bit.

size_t printHEX(uint16_t nr, uint8_digits)

{

  if (digits > 4) digits = 4;

  char buf[4];

  int i = digits-1;

  for (; i>=0; i--)

  {

    uint8_t n = nr & 0x0F;

    if (n <10) buf[i] = '0' + n;

    else buf[i] = 'A' - 10 + n;

    nr >>= 4;

  }

  uint8_t n = 0;

  for (i=0; i<digits; i++) n+= write( buf[i] );

  return n;

}

printHEX( char* ptr, uint8_t len)

{

  for (int i=0; i<len; i++)

  {

    printHEX( (uint32_t) *ptr++, 2);

    write(' '); // separator

  }

}

When not used the compiler will not link them...

- similar for printBIN() - 

Is this a better proposal?   

[Kristian Sloth Lauszus]

I think something like this would be better:

size_t printHEX(uint16_t nr, uint8_t digits) {

  uint8_t size = digits;

  do {

    char v = '0' + (((nr >> (digits - 1) * 4)) & 0x0F);

    if (v > '9') v += 7;

   write(v);

  } while(--digits);

  return size;

}

By not using a buffer, we are not limiting the number of digits.

Note that it is inspired by the USB Host library: https://github.com/felis/USB_Host_Shield_2.0/blob/master/printhex.h.

We can actually also just use the PrintBin from that as well.

Regards

Lauszus

[Rob Tillaart]

@Kristian, 

You are definitely right, it looks excellent to me, 

The reason why I used the buffer is to do less shifts, but a first order performance comparison shows equal speed (number with 4 digits = average case) 

while the footprint of your code is about twice smaller!

So lets take this as implementation of the 'workhorse' and add the overloaded functions for other types as proposed.

Thanks again for this perfect contribution,

Rob

 

[Matthijs Kooijman]

Hey folks,

to be honest, I'm not sure I actually like the "printHEX" API much, it's
quite specific compared with the more generic print(..., HEX) API. Also,
the name does not quite show that it actually does zero-padding, while
print() does not.

Previously, I think someone (Christian?) suggested adding a "digits"
argument to the existing print functions. e.g,.:

Serial.print(foo, HEX, 2);
Serial.print(foo, BIN, 8);

The default for the digits argument would be 0, meaning "as much as
needed".

To introduce the "automatic" version of this, you could pass 0xff as the
number of digits (nobody should ever want to print 255 digits):

#define AUTO_DIGITS 0xff

Serial.print(foo, HEX, AUTO_DIGITS);

This could be implemented using something like:

Print::print(uint8_t v, uint8_t base, uint8_t digits) {
if (digits == AUTO_DIGITS)
printNumber(v, base, 2);
else
printNumber(v, base, digits);

}


> > size_t printHEX(uint16_t nr, uint8_t digits) {
> > uint8_t size = digits;
> > do {
> > char v = '0' + (((nr >> (digits - 1) * 4)) & 0x0F);
> > if (v > '9') v += 7;
> > write(v);
> > } while(--digits);
> > return size;
> > }

I'm slightly worried about the bitshift in this implementation, since
the AVR hardware does not have a multiple bitshift instruction, so that
shift would be implemented as a loop of shifts. This would mean that in
total, the hardware would have to execute something like 2 * digits *
digits bit shifts, unless the compiler manages to simplify this.

Also, your implementation is hex-specific, but if we go for the
alternative API I suggested above, it should work for other bases as
well, meaning we should be changing the current printNumber
implementation instead of adding something new.

Gr.

Matthijs

[William "Chops" Westfield]

> I'm not sure I actually like the "printHEX" API much

Don't forget that avr-libc has printf and sprintf, which aren't that big compared to the memory of the modern CPUs.
Serial.print() was a nice simplification back on the ATmega8 for simple formatting needs, but maybe as we get into more traditional "requirements" like "hex with leading zeros" and advanced floating point output, we should stop reinventing the wheel and figure out how to use the standard tools/libraries for such purposes.

Can print(char* formatString, args, …) be made to work?

A more important enhancement would be to figure out how to pull in the float-point version of printf() when required…

WestfW

[Kristian Sloth Lauszus]

I agree with WestfW. I know Paul have recently implemented it for Teensy: https://github.com/PaulStoffregen/cores/commit/b4b56ae7470d786b703c1f283d6ec79ad2f61c93 maybe we should just add this to the Arduino core as well?

Regards

Lauszus

[Matthijs Kooijman]

Hey folks,

I actually like William's suggestion to include printf support: It is a
proven way to concisely specify number formatting, with the added
advantage that it can also easily print multiple items on a single line.

However, in Paul's commit, he comments:

> However, the Arduino Team has clearly stated they do not believe printf()
> should be part of the official Arduino API. Supporting printf() has been
> discussed many times on the Arduino developers mail list. The answer has been
> the same every time.

I couldn't find any of these discussions through the Google groups search
feature. Does anyone have a link for such discussions, or perhaps the
Arduino Team can comment?

Thinking about this, I can thinkg of two downsides:
- The printf code is big.
- Printf is, because of the varargs, susceptible to mismatching the
format string and the arguments passed, which could lead to undefined
behaviour. This is not so nice for novice users.

Regarding the code size, I guess that it's ok to pay some extra code
size for extra flexibility. Also, if you can replace all your prints
with printf, I think the extra code size might nog even be so much for a
typical sketch (but that should be measure to be sure).

Regarding the error-proneness, much of that can probably be avoided by
letting the compiler generate warnings for a mismatched format string
and arguments. This can be achieved using the "format" attribute, e.g.,:

__attribute__ ((__format__ (__printf__, 1, 2)));
http://gcc.gnu.org/onlinedocs/gcc-4.1.2/gcc/Function-Attributes.html#index-g_t_0040code_007bformat_007d-function-attribute-1791

Though that would require enabling warnings in the IDE, but that's
probably a good idea anyway...

> I agree with WestfW. I know Paul have recently implemented it for Teensy:
> https://github.com/PaulStoffregen/cores/commit/b4b56ae7470d786b703c1f283d6ec79ad2f61c93
> maybe we should just add this to the Arduino core as well?

Looking at that commit, it seems nicely implemented. It uses a custom
FILE* that writes to the Print object, which removes the need for a big
buffer like you would need when manually
using sprintf to write to a buffer first.

Gr.

Matthijs

[Tom Igoe]

I hate printf. It’s not so much concise as terse. It requires you to remember a number of non-standard meanings for punctuation symbols that have been standardized in many written languages long before C. It’s another one of those things that C does “well” from a programmer’s standpoint, but only if you’ve been trained in how to use it. I much prefer Matthjs (or was it Cristian’s, based on Dave’s original) print(value, BASE, digits)

t.

[Victor Aprea]

... if I had a nickle for every time I've had to write this code in a sketch:

if(value < 0x1000) Serial.print(F("0"));

if(value < 0x100)  Serial.print(F("0"));

if(value < 0x10)   Serial.print(F("0"));

Serial.print(value, HEX);

instead of: printf("%04xl", value);

... or things like ...

Serial.print(value1);

Serial.print(F("\t"));

Serial.print(value2);

Serial.print(F("\t"));

Serial.println(value3);

instead of: printf("%d\t%d\t%d\n");

Documentation of printf and format strings is very good and throrough. Maybe a compromise is just to have a library for advanced users akin to Java's java.util.Formatter class to expose the s(n)printf functionality.

On the other hand, there's nothing preventing advanced users from actually just using s(n)printf since it's just part of avr-gcc, right? Like this:

char tmp[32];

uint32_t value = 0xDEADBEEF;

snprintf(tmp, 32, "%04xl", value);

Serial.println(tmp);

So I guess in the end I see where Tom is coming from. "%04xl" is pretty nonsensical to the uninitiated (though I hardly think it merits hatred). If you need it you can use it already. As I see it,  print(value, BASE, digits) is ambiguous though. Principally, does it pad on the left with "0" or spaces? Maybe an optional fourth argument is in order at least (to override a default padding character).

Kind Regards,

Vic

Victor Aprea // Wicked Device

victor...@wickeddevice.com

M: 607.745.1515

T: @vicatcu

[Knight, Dave]

Good grief!  Aside from using some extra code space, what's the "problem" with standard C [s]printf() syntax?  Billions of programmers have been using it for decades, with few complaints! 

For me, as a C programmer from way back (c. 1982), and FORTRAN & assembler (c.1962) a single, simple sprintf(buf,"fmt", varList...)+Serial.print(buf) - IF it supported %[w[.p]]f format spec - would be far easier than having to code multiple sprintf()+Serial.print() calls for everything but floats and (totally different format-spec) Serial.print() calls, just to print one or more floats properly!

FWIW, having designed and written lots of OS/360, CP/CMS (VP/CSS, actually) and Unix system internals code, and even a small multi-tasking OS (PDP-11) from scratch, it seems to me that, if the code for Serial.print(var, precision) is already there, it shouldn't take much more code to support the sprintf() floating point functionality.

--
"Gun control empowers the wolves, and creates more sheep!"
              Dan Bongino, Former US Secret Service Agent

[Tom Igoe]

On Dec 29, 2013, at 12:09 PM, Knight, Dave <die...@gmail.com> wrote:

Good grief!  Aside from using some extra code space, what's the "problem" with standard C [s]printf() syntax?  Billions of programmers have been using it for decades, with few

complaints! 

Billions seems an exaggeration, with only 7 billion people on the planet.  And there have been many complaints, notably from people who drop out of programming because many languages, C included, are written with an assumption that you have a background like yours. That is not the primary audience for Arduino, in my opinion. From the beginning, we’ve tried to reach the folks who were turned off from programming for a variety of reasons, including the syntax, the complexity, and the fact that experienced programmers told them “Good grief!  If it’s good enough for me, you can learn it!” 

I’ve taught hundreds of people over the past dozen years or so who *don’t* come from programming backgrounds, or who gave up on learning it because they encountered the attitude you’re expressing here from their teachers, programmer friends, etc. And one of the most common complaints I hear is about terse syntax, of which sprint and regex are the canonical examples. 

I’m not denying their power. But I contend that they are not friendly to beginners. And that is who I care about. I feel that, given the extra programming power we have now, we should be able to “waste” a few cycles on more understandable syntaxes. If we were going to settle for a tool  because it’s a standard, there’d have been no need to create Arduino, or Processing, in the first place. For that matter, we might as well all stick with assembler, why make new languages?

For me, as a C programmer from way back (c. 1982), and FORTRAN & assembler (c.1962) a single, simple sprintf(buf,"fmt", varList...)+Serial.print(buf) - IF it supported %[w[.p]]f format spec - would be far easier than having to code multiple sprintf()+Serial.print() calls for everything but floats and (totally different format-spec) Serial.print() calls, just to print one or more floats properly!

Right, but what about for someone who’s *not* a C programmer?  If you had the opportunity to make a new language, given the current landscape and the audience we’re looking to address, how would you do it differently?  

[Matt Jadud]

On Mon, Dec 30, 2013 at 10:48 AM, Tom Igoe <t.i...@arduino.cc> wrote:

On Dec 29, 2013, at 12:09 PM, Knight, Dave <die...@gmail.com> wrote:

Good grief!  Aside from using some extra code space, what's the "problem" with standard C [s]printf() syntax?  Billions of programmers have been using it for decades, with few

complaints! 

The whole paper doesn't bare reading, but I'm happy to provide it if anyone is interested. 

Results from http://www.tandfonline.com/doi/abs/10.1080/08993408.2010.507335#.UsGyKGRDvV0

The summary: in a study of 143 students, engaging in their first assignment using format strings (in Java), 8% of all syntax errors and 100% of all exceptional run-time behavior were caused by errors in student use of format strings. 

Format strings are a language; as a language, format strings are embedded, they are unchecked by the compiler, and they are cryptic and confusing to novices (at best). Students are likely to get a format string "correct" as far as the compiler is concerned, but end up with odd or unpredictable runtime behavior.

Tom's question is the pertinent one:

"Right, but what about for someone who’s *not* a C programmer?  If you had the opportunity to make a new language, given the current landscape and the audience we’re looking to address, how would you do it differently?"

Cheers,

Matt

[Knight, Dave]

Yes, "billions" was an (intentional) exaggeration.  However, one thing I learned early on was that, when it comes to programming languages (machine language, assemblers, higher-level languages), they are all just dialects expressing the same fundamental constructs - essentially, if you know one, it is relatively simple to learn another.

Would you "give up" speaking because there are many languages?  Of course not; you'd learn the ones you need to achieve your goals!

Since 1975, I, too, have taught many "beginners", from school children & college students to retired EEs, to program in various languages including BASIC, FORTRAN, COBOL, assemblers, C, C++, AWK, PERL, etc. on various hardware and OS platforms.  Seldom has my "attitude" (which I prefer to think encourages learning new things) "offended" anyone except perhaps a few TelCo "Bell Heads" inured to the notion that having learned COBOL in the 70s, and achieving a certain age/job-title/tenure they should never have to learn anything new  ;-)

Even for experienced programmers, the Arduino "sketch" can be a challenge; not because of the C/C++ language, but because doing non-trivial things in a function that gets called maybe 140,000 times per second (on an Uno) requires a mindset wholly different from traditional procedural programming!

Regarding:

For me, as a C programmer from way back (c. 1982), and FORTRAN & assembler (c.1962) a single, simple sprintf(buf,"fmt", varList...)+Serial.print(buf) - IF it supported %[w[.p]]f format spec - would be far easier than having to code multiple sprintf()+Serial.print() calls for everything but floats and (totally different format-spec) Serial.print() calls, just to print one or more floats properly!


Right, but what about for someone who’s *not* a C programmer?

Do you really believe that the convoluted code made necessary by the lack of a working %f format spec is EASIER for a non-C-programmer or a "beginner"?

I have no desire to invent yet another language, or to dumb-down an existing language, simply to attract "beginners" who, for whatever reason, have no interest in learning to program a computer.   What would such individuals hope to accomplish with an Arduino?  Frankly that seems like a fool's errand, akin to the (facetious) 1960s quest for the elusive "DoJob" macro  ;-)

If not folks who want to program an Arduino to "do something" of their own invention, who, exactly, do you see as the target audience for Arduino?

FWIW, I disagree entirely with your assertion: "If we were going to settle for a tool  because it’s a standard, there’d have been no need to create Arduino, or Processing, in the first place."  The Arduino provides an inexpensive and very capable platform that opens up a vast array of (small) process monitoring & control opportunities that had previously been unavailable except on much more expensive, special-purpose or custom process control systems, most utilizing extremely arcane languages and codes (e.g. CNC)..

The Arduino IDE, by embracing the ubiquitous C/C++ language, makes this kind of technology readily available to anyone with problem-solving capabilities and the desire to learn, and invent!

The sheer size of the Arduino "community", indicated by the number of contributors to C/C++ libraries, code projects and forums suggests that there are "millions" (perhaps another exaggeration  ;-) of folks who have ventured past whatever difficulties they may have had with C/C++.

That said, I reiterate my desire that Arduino's sprintf() support the standard format syntax, perhaps as a library/option to conserve program space for those who don't need it.

Dave

[Matthew Ford]

For my two bits I think this is the most important point (thanks Matt)

"Format strings are a language; as a language, format strings are
embedded, they are unchecked by the compiler, and they are cryptic and
confusing to novices (at best)"

More compiler checking is better.
matthew

On 31/12/2013 5:05 AM, Matt Jadud wrote:
> On Mon, Dec 30, 2013 at 10:48 AM, Tom Igoe <t.i...@arduino.cc> wrote:
>
>> On Dec 29, 2013, at 12:09 PM, Knight, Dave <die...@gmail.com> wrote:
>>
>> Good grief! Aside from using some extra code space, what's the "problem"
>> with standard C [s]printf() syntax? Billions of programmers have been
>> using it for decades, with few
>>
>> complaints!
>>
>> The whole paper doesn't bare reading, but I'm happy to provide it if
> anyone is interested.
>
> Results from
> http://www.tandfonline.com/doi/abs/10.1080/08993408.2010.507335#.UsGyKGRDvV0
>
> The summary: in a study of 143 students, engaging in their first assignment
> using format strings (in Java), 8% of all syntax errors and 100% of all
> exceptional run-time behavior were caused by errors in student use of
> format strings.
>
> Format strings are a language; as a language, format strings are embedded,
> they are unchecked by the compiler, and they are cryptic and confusing to
> novices (at best). Students are likely to get a format string "correct" as
> far as the compiler is concerned, but end up with odd or unpredictable
> runtime behavior.
>
> Tom's question is the pertinent one:
>

> "Right, but what about for someone who�s *not* a C programmer? If you had

> the opportunity to make a new language, given the current landscape and the

> audience we�re looking to address, how would you do it differently?"
>
> Cheers,
> Matt
>

[follower]

On 31 December 2013 07:32, Knight, Dave <die...@gmail.com> wrote:
> I have no desire to invent yet another language, or to dumb-down an existing
> language, simply to attract "beginners" who, for whatever reason, have no
> interest in learning to program a computer. What would such individuals

> hope to accomplish with an Arduino? [...snip...]

>
> If not folks who want to program an Arduino to "do something" of their own
> invention, who, exactly, do you see as the target audience for Arduino?

I'm not aware if there's a more recent document but it's worth
visiting the Development Policy document
<https://code.google.com/p/arduino/wiki/DevelopmentPolicy> that was
created when a similar discussion occurred on this mailing list a few
years ago.

Specifically:

"""
Arduino aims to make technology accessible and useful to new audiences
such as artists, designers, or hobbyists. The development efforts
focus on that which allows people to get their work done. To that end,
it strives for the dual goals of simplicity (so people can actually
use the platform) and robustness (so they create things that work).
"""

The drive for this audience is to accomplish some project not learn to
program a computer.

That the Arduino has also attracted a technically sophisticated
audience is a "happy" accident but as has been reiterated over the
years the target audience is still the non-technical person and their
experience will be prioritised over any others.

--Philip;

[Victor Aprea]

What are we even talking about at this point?  Is there a proposal on the table?

Remember, as I noted in my last post,  it's all avr-gcc under the hood (so far as AVR cores go) so feel free to use standard libs to your heart's content...

Regards,
Vic

[Tom Igoe]

Dave, I’ve offended you. I’m sorry, that was not my intention. The term “turned off” was not meant to imply “offended”. And “hate” was too strong a term. Let’s just say I’m one of the subjects mentioned in the paper that Matt Judud cited (Matt, I’d love the PDF if you have it).

I’m happy to explain my assumptions in more depth if you wish, but we can take it offline if so. As Philip points out, I’ve done that many times before, and others are tired of hearing it. Meantime, back to the topic:

As Vic pointed out, you can use the standard libs all you want, so format strings are theoretically already available. Given that, is there need for something else? If so, what?

Tom

[Paul Stoffregen]

What are we even talking about at this point?  Is there a proposal on the table?

I believe it's about supporting printf() in Arduino's Print class.  Or whether printf and format strings are appropriate for Arduino....

Remember, as I noted in my last post,  it's all avr-gcc under the hood (so far as AVR cores go) so feel free to use standard libs to your heart's content...

Certainly you can use C library functions to write this:

  char buf[20];
  sprintf(buf, "data=%08lX\n", myint32);
  myLCD.print(buf);

You can also write:

  myLCD.print("data=");
  myLCD.println(myint32, HEX, 8);  // well, you could if Arduino supported this.....

However, experienced programmers who love printf() want this:

  myLCD.printf("data=%08lX\n", myint32);

I believe the experienced programmer viewpoint is this last method is 2X or 3X better than the first two, because it uses 1 line in the source code rather than 2 or 3.  It's probably *MUCH* less efficient at runtime, since varargs use the stack instead of fast registers to pass parameters, and printf() does extra work at runtime to decode the format string, and on AVR it can't take advantage of the Print class block write optimizations (on hardware that supports block write).  I could be wrong, this might be about more things, but I suspect the one and only thing that really, really matters to people want printf() is the number of lines they have to write into the program.  It's not about memory or cpu usage, but vertical real estate in the source code, and perhaps the familiarity of a widely used (but complex) syntax.  In that world view, one line with terse error-prone syntax (eg, needing "lX" instead of "X" for 32 bit integers) always beats 2 or 3 lines, no matter how simple and highly readable the multi-line approach might be.

Over the last 4 years, I've heard this request (regarding Teensy) over and over again.  Daily would be an exaggeration, but it has been requested very frequent.  Teensy tends to appeal to more technically included users than official Arduino boards, so it's quite likely the feedback I've heard isn't really representative of the larger Arduino community.  About a month ago, I finally added it to Teensy's core library.

I'm happy to contribute this code.  You can help yourself to it via github or grab it from hardware/teensy/cores after running my installer.

However, the wisdom (or lack of wisdom) to include printf support in Arduino has been discussed here many times over the years.  It's pretty much always the same... Tom and people who value simplicity and novice-readable code vs experienced programmers who want 1-line syntax and a familiar feature, regardless of the complexity.

The last thing I intended to do was reopen this discussion yet again.  I believe you can see that in my comments on the github commit.  I'm definitely not advocating this feature one way or the other... just trying to clear things up a bit.  Hopefully this helps?

[Victor Aprea]

Thanks Paul. What is the downside to adding a printf method to the Print class that simply exposes / wraps the stdlib sprintf method? Beginners need not use it, and for advanced users it can hide those three pesky lines of code / buffers in the object.

Regards,
Vic

[Paul Stoffregen]

On 12/30/2013 03:08 PM, Victor Aprea wrote:

Thanks Paul. What is the downside to adding a printf method to the Print class that simply exposes / wraps the stdlib sprintf method? Beginners need not use it, and for advanced users it can hide those three pesky lines of code / buffers in the object.

Well, the likely downside for novice users is more people will tend to publish examples on the playground and numerous websites using complex syntax like Serial.printf("%-lu",u32) rather than simple syntax like Serial.print(u32).  Novices depend heavily upon easy examples and well designed libraries.

Then again, some people already publish widely used libraries with examples using printf (notably maniacbug) and the C++ streams syntax (notably sdfatlib).  The reality is some of the very best and most widely used Arduino library are developed by experienced programmers who favor the compactness (in lines of source) printing formats, even in their examples meant for novices.

[Tom Igoe]

On Dec 30, 2013, at 7:17 PM, Paul Stoffregen <pa...@pjrc.com> wrote:

Well, the likely downside for novice users is more people will tend to publish examples on the playground and numerous websites using complex syntax like Serial.printf("%-lu",u32) rather than simple syntax like Serial.print(u32).  Novices depend heavily upon easy examples and well designed libraries.

Then again, some people already publish widely used libraries with examples using printf (notably maniacbug) and the C++ streams syntax (notably sdfatlib).  The reality is some of the very best and most widely used Arduino library are developed by experienced programmers who favor the compactness (in lines of source) printing formats, even in their examples meant for novices.

Needless to say, I’m in favor of those examples that use the simple syntax over the compact. It’s a problem when a student can’t understand a line of code in a sketch, and that’s the line they need to change. And the example writer didn’t put in a comment explaining the code, because they feel the user should RTFKernighan&Ritchie.

t.

[Knight, Dave]

I am pleased to read all the thoughtful discussion here. 

My issue with Arduino's sprintf(...) implementation is simply that standard C library printf() format specs (e.g. %f), that compile without error using the Arduino tool chain, should work, but they do not, and the suggested "work-around" requires extra hand-coding and function calls to get the correct output.

Tom, I am not offended, and I'm not even sure we disagree.  Help me out here: I am (singularly?) unaware of any accommodations in the Arduino universe that would enable "artists, designers & hobbyists" to use an Arduino without first learning some C.  Surely no one seriously considering the Arduino platform believes they can make their application work by "wishing upon a star".

For the record, I consider APL a "write-only" language for the very issue you raise: "concise" vs. simple syntax.  APL programs, unless you use APL frequently, soon become unreadable ... omnia mihi lingua graeca sunt - literally  ;-)   But C is hardly in the same class as APL!

A line or block of code I don't understand prompts me to figure it out and maybe even crack a book, if necessary.  I would expect libraries and sample code to set a good example and promote best practices - from which others can learn. 

Furthermore, in my experience, folks introduced to programming thru a "dumbed-down" interface, soon tire of its limitations and/or awkward constructs and demand more flexibility.  Witness the evolution of the original BASIC language into Visual Basic.  Usually it's easier (and quicker) to learn a language that better suits the task at hand, than to wait for a simple tool to evolve into something more useful.  But I digress...

Regarding "RTF-K&R", why not RTF-Arduino Documentation?  It effectively covers the same ground, with some additional Arduino specifics.

Arduino applications are C/C++ programs.  Calling them "sketches" does not change that. 

Dave

--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[Matthijs Kooijman]

Hey folks,

let me first summarize the arguments in this discussion, roughly divided
into arguments for printf (+) and arguments against (-).

+ Code for printing values is more terse.
+ If you know how printf format strings work, printf is more
readable than half a dozen print calls.
- Format strings are not very intuitive, novice users will probably
have no idea what they mean.
+ printf is a very standardized format, so there is plenty of
documentation.
+ Offering both the existing print methods and the new printf
method allows novice users to use print and more advanced users to
user printf if they want.
+ When the printf method is not used, it is not linked in and it will
not use any code size.
- Offering printf might mean that example code will more often use
printf, making it more confusing for users.
- printf does not support floating point numbers by default, it needs
extra compiler options to support those.
- printf format strings are more error-prone (errors at runtime).
+ Adding printf to the Print class makes it easy to use it. A sketch
can always use sprintf with a buffer and print the buffer, or duplicate
the code Paul wrote for Print::printf into a separate printf function,
but that's less clean and compact.


Did I miss any arguments?


Since I'm in favor of adding printf, I'll comment on the downsides:

- Format strings are not very intuitive, novice users will probably
have no idea what they mean.
- Offering printf might mean that example code will more often use
printf, making it more confusing for users.

These two are related. The first downside by itself is not really a
problem, novice users can simply avoid printf, unless they are "forced"
to look at it because some example code they need uses it.

I consider this the biggest problem for which I have no perfect retort,
other than to hope that people writing examples will consider not using
printf because of this very reason.


- printf does not support floating point numbers by default, it needs
extra compiler options to support those.

By default, gcc links against a vprintf implementation that does not
support floats. Some linker options are needed to let it use the full
vprintf implementation. We could add an option for this to the IDE,
allow the user to specify arbitrary compiler options, or just accept
that floats are not supported - even without float support printf still
adds a lot of value.

Note that when %f is disabled, printf just prints "?" instead of the
floating point value, so at least it does not crash in horrible ways :-)


- printf format strings are more error-prone (errors at runtime).

This is true, but a lot can be mitigated by letting gcc emit warnings
for invalid format strings (which already happens in 1.0.x in verbose
build mode, see also my other thread about warnings).

One problem I did see is that the compiler does not warn about the use
of %f when it is not available. Since enabling printf float support
happens at link time, there really isn't any way for gcc to
automatically know when to warn about this either. In theory, it could
have a -Wformat-float flag to allow the IDE to enable these warnings
when it's not enabling printf float support, but it seems such a
-Wformat-float option is not enabled. The IDE could solve this by
scanning the source for %f used in printf format strings and adding its
own warning about this, which could be a good solution (note that I'm
_not_ saying to automatically enable float support in this case, since
that would be tricky to get right...).



Gr.

Matthijs

[Tom Igoe]

I think your hope here is unrealistic, Mattijs. Examples tend to be written by more experienced users, and they tend to write using the methods they know. If you add printf, you will see more examples using it. And since experienced users tend not to be good at comments, you’ll see less explanation of them. That’s my guess based on what I’ve seen so far.  

Now, if you could add a pre-compiler directive that spat out “Come on, you’re going to put in a line like 'printf( "a = %5.5d\\n", a );’ without an explanatory comment??? I’m not compiling that until you explain it!”, I’d be happier.

t.

[Knight, Dave]

I would only add that "best practices" for contributed libraries and sample code should include comments adequate for moderately skilled users.  While that's pretty subjective, I suspect most folks skilled enough to submit libraries and sample code can make reasonable decisions on what needs to be commented.

Since many people can learn advanced techniques more quickly from good examples, than from sometimes tedious texts, I believe well-written  libraries & sample code offer great value to non-expert and advanced users alike. 

Otherwise, your analysis & suggestions work for me. 

Thanks!

Dave

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAlLCt28ACgkQz0nQ5oovr7x0DQCggddutACInfr+a7PQFu1Ax33l
+1MAoIHpOaY22WrDEBhTPgktFKO4UQby
=INok
-----END PGP SIGNATURE-----

[Tom Igoe]

"Moderately skilled" is a slippery slope. Broken record here, but I would say all sample code needs to include comments for users who know nothing more than the core functions of Arduino. I think if you assume anything more than that, you need to point the user to a good reference to learn the concept you are using.

[Victor Aprea]

Dave,

For printf (instead of sprintf) to work "out of the box" you just have to implement putc and bind it to the UART (or whatever)... anyway, for the sake of awareness I just want to point out the existence of http://playground.arduino.cc/Main/Printf

Regards,

Vic

Victor Aprea // Wicked Device

victor...@wickeddevice.com

M: 607.745.1515

T: @vicatcu

[Rob Tillaart]

Major point in this discussion - between the lines maybe - is the level (or lack of) of documentation. Comments in code, variable and function names, playground articles and otherwise is the real problem. If there is no good documentation one can only learn it the hard way.

I think the documentation on a wiki e.g. - http://wiki.pinguino.cc/index.php/AnalogRead - that is modifiable by more than a few would be a great step forward.

Then it is possible to document something as printf(); and explain it. 

Improved documentation would be a nice goal for 2014!

-------------------------------

I agree with Tom Igoe that Arduino should be usable for everyone without skills and willing to learn. 8-10 year old kids should be able to start tinkering.

I am not against support for printf() - lets just do it, there is a clear need for it, don't spend energy on the discussion, built it. But IMHO we should really spend energy to keep the simplified interface easy to use for beginners and powerful enough for the more skilled. Defaulting parameters in the print class (and every other one) is imho a good way to go.

print(float)  -> prints automatically with 2 decimals (beginners mode)

print(float, 3) -> learned an extra parameter for # digits

print(float, 4, '0') -> Learned an extra parameter for leading characters 

print(float, 4, '0', SCI) -> learned an extra parameter for scientific notation

Simplifying an interface is hard and yes, we have seen people for who the "simple"  print(float, decimals) is a pattern that they replicate in

float x = y,2;  // x equals y rounded to two decimals.

And from their point of view it makes sense ...

my 2 cents

Rob

And for the new year: I hope that all your dreams get wings !

[Tom Igoe]

On Dec 31, 2013 10:23 AM, "Rob Tillaart" <rob.ti...@gmail.com> wrote:
>

>
> Improved documentation would be a nice goal for 2014!
>
> -------------------------------
>
> I agree with Tom Igoe that Arduino should be usable for everyone without skills and willing to learn. 8-10 year old kids should be able to start tinkering.
>

Just thinking out loud, one way to make this happen would be to require that all submitted patches include documentation that conforms to the guide for docs and is reviewed by someone from the teacher's list....

[Matthijs Kooijman]

Hey Folks,

I've changed the subject of this mail since we've drifted very much off
the original topic. However, we've now touched upon something about
which I've wanted to write something for a while now, so this seems like
a good moment to do so.

> > Improved documentation would be a nice goal for 2014!
> >
> > -------------------------------
> >
> > I agree with Tom Igoe that Arduino should be usable for everyone without
> > skills and willing to learn. 8-10 year old kids should be able to start
> > tinkering.
> >
>
> Just thinking out loud, one way to make this happen would be to require
> that all submitted patches include documentation that conforms to the guide
> for docs and is reviewed by someone from the teacher's list....

As far as I know, the main documentation is currently the website.
However, there is no easy way for people to contribute documentation for
the website, since AFAIK it can only be edited by official developers,
or whoever.

Accompanying a patch for a new feature with some documentation can work
if it's a matter of adding a new paragraph to the documentation, but
more complicated changes are hard ("Then change this paragraph to say
<this> and the last sentence of that page should be <that>").


Ideally, the (source files for the) documentation on the website would
also live in a git repository and pushes to the repository would
automatically be (built and) published.

I'm really unsure what format and system / CMS is used for the website
now, so this might not be easy to implement, but I think it could very
much improve the documentation if people can easily submit patches /
pullrequests with documentation updates. If this is the case, then it's
also a lot easier to respond to a code patch with "Patch is good, please
submit a documentation patch as well and we will pull both".


As an example, I've seen this system work very well for uwsgi, a web
application server. They have a repo here, containging sphinx /
restructured text source files:

https://github.com/unbit/uwsgi-docs

Pushes to that repository get automatically published onto their
readthedocs.org page:

https://uwsgi-docs.readthedocs.org/en/latest/


Perhaps something like this could work for Arduino as well?

Gr.

Matthijs

[Matthijs Kooijman]

Hey Victor,

> For printf (instead of sprintf) to work "out of the box" you just have to
> implement putc and bind it to the UART (or whatever)... anyway, for the
> sake of awareness I just want to point out the existence of
> http://playground.arduino.cc/Main/Printf

Overriding putc (and thus printf) to print to Serial is nice, but it's
limited - you'll likely want to use printf on other Print objects as
well (secondary serial ports, ethernet clients, etc.).

The patch that originally started this printf suggestions solves this by
implementing a Print::printf method:

https://github.com/PaulStoffregen/cores/commit/b4b56ae7470d786b703c1f283d6ec79ad2f61c93

which seems to me like the proper way to solve this. Alternatively, a
printf function that just takes a "Print" object as its first argument
could be a fair compromise:

printf(Serial, "foo: %s", foo);

(though you might need to use a different name than "printf" to prevent
lnker errors, not sure).

Surprising that this approach isn't suggested on the playground page as
well....

Gr.

Matthijs

[Georgitzikis Vasilis]

ah, good point.

a good solution for that (and something we wanted to contribute to as codebender for a while, just caught up in various stuff), is to simply use doxygen to document the public Arduino core and library functions. you can then not only create a doxygen html-based documentation with one command (which you can style as you want with css), but you can also generate the intermediate xml-based documentation that doxygen supports, which you can then parse and output to any type you want.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAlLC6jcACgkQz0nQ5oovr7wdHACgzNvshWw9C44oDuaOtpdbirVq
JSIAoJp4NNL3QP4tqJUeeo0JOX6EATJy
=I2KU
-----END PGP SIGNATURE-----

[Tom Igoe]

I’d appreciate it if you also take this to the arduino-teachers list, as there are many useful and less technical voices there. I’ve cc:ed the teachers list here. 

I’ll also add some details you may not have been aware of.

For reference, we have been working on documentation a good bit the last few months, though I don’t know if it’s noticed.  There is a style guide for docs here:

http://arduino.cc/en/Reference/StyleGuide

And a style guide for APIs here:

http://arduino.cc/en/Reference/APIStyleGuide

And Scott Fitzgerald, who wrote the Starter Kit booklet, is now responsible for docs, along with Federico Vanzati. If you haven’t read the starter kit booklet, I recommend you check it out. Scott did a really great and thorough job, and Michael Shiloh did a nice job editing and commenting on it to improve it. I used it as part of my teaching materials this past fall, and it was very useful.

Scott and Fede now have a template for documenting all new products, which includes a Hardware Page, a Getting Started guide, a reference section for any new libraries, and example pages for all new examples written. Starting a year or so ago, we made a push to make sure that every example included in the core or the core libraries has an example page on the site explaining it. And we make sure any new libraries have examples, written following the style guide.  

The folks at Temboo used this material when they wrote their examples for the new Yun board, and I’ve gotten a number of compliments on it from users. The Temboo team were really great to work with, and I think they’d be a useful part of the conversation too.

The reference section is probably the part that could use the most improvement, as it’s the most terse section of the site, in terms of writing style. They’re also working on improving the explanation around new examples.  We also need to make sure all the examples have links in their headers to the example pages, now that those links are clickable.

Scott and Fede haven’t been able to return to older documentation to shore it up, unfortunately, as the workload for new products has taken all of their time thus far. We hope that will change in the near future. However, they have a pretty thorough to-do list, and I’m sure they’d be happy to pass some of those on to volunteers.

Tom

[Tom Igoe]

Sorry, forgot to add:

Given all of that, it would be helpful to hear what specific things would improve documentation. What would make it more readable, more understandable by beginners, more useful on an on-going basis? What’s not explained that could be? What’s explained that isn’t necessary? We know there are plenty of tools we could use, but good tools and bad writing still = bad docs.  Let’s talk what would improve the explanations.

t.

[Rob Tillaart]

Recurring complaints on the forum about the documentation are:

- not all code examples are up to date.

  A problem not mentioned yet is versioning of the documentation. 

   Does code/doc apply to pre 1.0, 1.01, 1.05, 1.55 etc?

- contain errors in the description - also errors in the hardware proposed

  

- completeness, which may be consequence of the "version problem"

As an author of a few playground articles I know it is hard to write good, readable and without errrors (you see;)

Imho it would be a good step to have some additional people (not necessary the moderators of the forum) that can do minor updates in the reference and tutorial sections to improve them. 

Two examples:

The description of delay() includes a link to blink without delay [which is good!], although I think that example should be on the delay page itself. And yes a more elaborated blink (multiple leds) without delay on another page. Maybe a page "How to schedule multiple actions with Arduino"

The millis() pages tells it overflows in 50 days. It does not tell how to handle that overflow in an appropriate way (subtraction iso addition). It also does not tell that millis() can skip a value due to its current implementation, and that in general it is better to use inequalities "> >=  =< <" to compare to millis() and micros(). There is an error in  "note that the parameter for millis is an unsigned long", which should be "note that the return value of millis() is an unsigned long". It is a minor error but still .

Note delay() has a section called Caveat, every reference page should have it (default "no known").

Diner time :)

[Knight, Dave]

Victor,

Today is a good day;  I  learned something new!  I was unaware of the "fdev_setup_stream()" capability.

Unfortunately, this does not solve the problem of missing %f library support.  Some have suggested that one must "pass extra flags to gcc", e.g.:

   -Wl,-u,vfprintf -lprintf_flt -l



which I would gladly do, but thus far, I have not found a straightforward way to "pass flags to gcc" (e.g. an option in preferences.txt), in the Arduino Docs or elsewhere.

Dave

[Knight, Dave]

Thanks, folks, for all the suggestions!

Much as I hate to admit that I'm stumped, I cannot find a simple method, using the current Arduino IDE, way to pass flags to the compiler, e.g. flags that might cause a sketch to be linked with the (provided)  libprintf_flt.a library.

I don't really want to build a custom tool chain, modify library content, create a customized IDE, or move away from the Arduino IDE by rolling my own build scripts.

That said, about the best that I've been able to do is admittedly a kludge, but it is almost straightforward, it is (sort of) within the printf() paradigm, and it does work.

Specifically, to print a float in the printf() context, use the %s format spec and move the "formatting" responsibility to the variable list with a call to:
  

char* dtostrf	(	double	__val,
		signed char	__width,
		unsigned char	__prec,
		char *	__s
	)

For example, to get the printf statement in the fdev_setup_stream example in the Playground:

   printf("Alive %.3f sec\n", seconds ) ;

to print the value of seconds properly, do something like:

   printf("Alive %s sec\n", dtostrf(seconds,3,3,buf) ) ;

The following lines:

   ...

   char buf[20];  // a buffer for the dtostrf() conversion

   ...
    // report seconds since starting
    printf("Alive %.3f sec\n", seconds ) ;
    printf("Alive %s sec\n", dtostrf(seconds,0,3,buf) ) ;
   ...

substituted in the fdev_setup_stream example, yield the following output to the serial monitor port (captured with minicom, as the Arduino serial monitor does not let me copy from its display):

################  Wed Jan  1 14:36:59 EST 2014
Alive ? sec
Alive 5.002 sec
Alive ? sec
Alive 6.002 sec
Alive ? sec
Alive 7.003 sec
Alive ? sec
Alive 8.003 sec
Alive ? sec
Alive 9.004 sec
Alive ? sec
Alive 10.004 sec
Alive ? sec
Alive 11.004 sec
Alive ? sec
Alive 12.005 sec

[follower]

On 1 January 2014 05:15, Tom Igoe <t.i...@arduino.cc> wrote:
> If you haven’t read the starter kit booklet, I recommend you check
> it out.

I tried to find the booklet online but it seems like only this sample
is available: <http://arduino.cc/en/ArduinoStarterKit/Prj02>?

--Philip;

[Matthijs Kooijman]

Hey Tom,

> Given all of that, it would be helpful to hear what specific things
> would improve documentation. What would make it more readable, more
> understandable by beginners, more useful on an on-going basis? What’s
> not explained that could be? What’s explained that isn’t necessary? We
> know there are plenty of tools we could use, but good tools and bad
> writing still = bad docs. Let’s talk what would improve the
> explanations.

I personally don't have any particular pet peeves or problems with the
documentation. Overall, I think it's good - but occasionally I come
across a small problem. When the documentation is really incorrect, I
usually (recommend to) submit a ticket on github to get it fixed, but
when the documentation is slightly incomplete or vague, I usually don't
bother creating a ticket trying to explain what would need to be
changed.

If I could just improve the documentation myself and submit a
pullrequest, I'd be more inclined to actually invest some time in it.


So, any comments to my original suggestion - to put the documentation
in a git repository?

> > I’d appreciate it if you also take this to the arduino-teachers
> > list, as there are many useful and less technical voices there. I’ve
> > cc:ed the teachers list here.

It seems you didn't actually CC the teachers list?

Gr.

Matthijs

[Matthijs Kooijman]

Hey Georgitzikis,

> a good solution for that (and something we wanted to contribute to as
> codebender for a while, just caught up in various stuff), is to simply use
> doxygen to document the public Arduino core and library functions. you can
> then not only create a doxygen html-based documentation with one command
> (which you can style as you want with css), but you can also generate the
> intermediate xml-based documentation that doxygen supports, which you can
> then parse and output to any type you want.

Normally, I'm all for doxygen-based documentation. However, in this
case, I see two problems:
- All of the existing documentation would need to be merged back into
the source code, which is a lot of work.
- The current reference documentation doesn't follow the actual code
structure 100%. In a lot of cases, the presentation of the reference
documentation is tweaked so it is clear to novice users, who are
unaware of concepts like classes, attributes and methods. When you
generate docs with doxygen, it's probablly inevitable that these
concepts will appear in the (structure of) generated documentation,
which makes the docs harder to read for novice users.

I haven't actually _tried_ to mimic the current documentation
structure with doxygen, so this is just an expectation.

Gr.

Matthijs

[Tom Igoe]

The booklet’s not online, it’s part of the starter kit. But yes, that is a sample project from it.

t.

[Tom Igoe]

Will discuss the git repo idea with Scott, Fede, and Roberto, our webmaster. It’ll take a little work, I suspect. We have an internal version of the docs that we use for editing and review before we publish, and any git version would have to sync with that as well.

I had cc:ed teachers on the first mail, not sure why it bounced.

t.

[Matthijs Kooijman]

Hey Tom,

> Will discuss the git repo idea with Scott, Fede, and Roberto, our
> webmaster. It’ll take a little work, I suspect. We have an internal
> version of the docs that we use for editing and review before we
> publish, and any git version would have to sync with that as well.

Cool, thanks!

Regarding the internal version - sounds like that would just be a
separate branch in git, but you'll have a better view of things to
figure out how to do this.

Gr.

Matthijs

[Tom Igoe]

Roberto will have a better view, actually, and I will defer to him on it, since he manages the web content.

t.

[Matthijs Kooijman]

Hey Tom,

> And a style guide for APIs here:
> http://arduino.cc/en/Reference/APIStyleGuide

Just now, I was looking for this particular page, but couldn't find it
by looking around the Arduino site or Googling. Two places I looked and
where I think it might be useful to link to this page are:

http://arduino.cc/en/Hacking/LibraryTutorial
and perhaps http://arduino.cc/en/Reference/Libraries

Perhaps someone could add a link there?

Gr.

Matthijs

[Tom Igoe]

Good idea. I'll makes those links when I'm on a keyboard.

T.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAlLLwRMACgkQz0nQ5oovr7wjcgCfaIjrdAGLEpas3mg9Dmt8j4xv
VkYAoOJcqL1ppe64JHnS+I56uhRCSPS5
=xaEZ
-----END PGP SIGNATURE-----

[Tom Igoe]

Links added to both of the documents Matthijs suggested.