Print enhancement available for review

[Michael Jonker]

Hello,

The implementation of the Print enhancement is available for review.
This thread is a follow up of the thread "Adding format control to the Print class methods";

You can find the implementation in my github fork /MichaelJonker/Arduino in the branch review_FormattedPrint :
i.e. https://github.com/MichaelJonker/Arduino/tree/review_FormattedPrint

The relevant files in here are under hardware/arduino/avr/cores/arduino (location conforms to release 1.6)
PrintFormat.h   // defines the print format control classes (including the implementation of all methods)
Print.h         // defines the print interface (and inline implementations of trivial Print methods)
Print.cpp       // implementation of the non trivial Print methods

In short, this code will allow the user to code things like:

Serial.println(3.141592, PRECISION(4) & FIELDSIZE(3));          //   3.1416 ; note FIELDSIZE only concerns integer part, not the fraction
Serial.println(66, HEX);                                        // 42 ;
Serial.println(22, RADIX(5) & FIELDSIZE(8) & ALIGNLEFT);        // 42       ;  
Serial.println(34, OCT & FIELDSIZE(8) & FORCESIGN & FILLZEROS); // +00000042 ; note forced sign does not consume FIELDSIZE
Serial.println(54, 13);                                         // 42 ; backward compatible base argument
Serial.println(3.141592, 3);                                    // 3.142 ; backward compatible precision argument

There are some more handy features. These can be discovered from the sample code stored in the above mentioned github branch under examples/test_FormattedPrint
testFormattedPrint.ino      ino sketch which calls PrintDemo().
PrintDemo.cpp               demo showing the usage of the Print and PrintFormat classes.

Notes:
- The file PrintDemo.cpp contains instructions to compile and run the PrintDemo in standalone mode outside the Arduino environment (I used this under Cygwin for a fast development cycle)
- The file PrintSequenceTeaser.cpp, also present, is a teaser (aka preview) of a PrintSequence class (still in the design stage).
  (However, please keep this thread focussed on the Print and PrintFormat classes and do not comment features related to the PrintSequence class in this thread.
  Discussion related to the PrintSequence class will be launched in different thread soon.)

The Print and PrintFormat classes are functionally complete. I plan to work on optimisation (code reduction) and the implementation of a scientific print format.
I have chosen for the print format syntax in the style: OCT & FIELDSIZE(6) & FILLZEROS.
A syntax style like OCT.withFieldSize(6).withFillZeros() has not been provided (but could be added if this is what the community want).

I have not implemented a PrintFormatter class for custom formatting.
The primary reason is that one cannot define templates for virtual methods making a hypothetical abstract PrintFormatter class in first instance boring and inefficient (large vtable).

Furthermore, such a class makes only sense for basic types, non basic types need to implement a Printable interface for them to become printable. Hence there is already full control over the format.
With the latter in mind, one can easily see that custom formatting of basic types can be readily achieved through an adapter class:

//  Using an adapter class IPAddressFormatAdapter
    class IPAddressFormatAdapter : public Printable
    {
      typedef unsigned char IPAdress[4];
      IPAdress myIPAddress;
      static unsigned long& _asLong(IPAdress& anIPAdress) { return (unsigned long&) anIPAdress;} // interpret an unsigned long as a unsigned char[4]

      public:
      IPAddressFormatAdapter(unsigned long anIpAddress) { _asLong(myIPAddress)=anIpAddress;}

      size_t printTo(Print& out) const
      {
        unsigned char nc=0;
        for(int i=0;i<4; i++) {if(i!=0) nc+=out.print('.'); nc+=out.print(myIPAddress[3-i]);} // I'm an endian, I'm a little endian...
        return nc;
      }
    };

//  the following hypothetical (because not available) method invocation:
//  Serial.print(0x21428418), myIpAdressFormatter);
//  may be implemented as:
    Serial.print(IPAddressFormatAdapter(0x21428418));

In particular I would like your advise/comments on:
- is it acceptable that all PrintFormat constants are part of the namespace ArduinoPrint, and that this name space is automatically 'used' by including Print.h
  I would prefer a nested namespace like using Arduino::Print, but this change can be made later.
- are the chosen PrintFormat constants acceptable? i.e.: BIN, OCT, DEC, HEX, RADIX(n), FILLZEROS, PRECISION(n), FORCESIGN, FIELDSIZE(n), STRICTSIZE, ALIGNLEFT
- is the chosen PrintFormat merge operator& acceptable (e.g. HEX & FILLZEROES)
- is there a need for method based format modifiers: .asBin(), .asOct(), .asDec(), .asHex(), .withRadix(n), .withFillZeros(n), .withPrecision(n), .withForceSign(n), .withFieldSize(n), .withStrickSize(), .AlignedLeft()
- is it acceptable that FORCESIGN and PRECISION(n) are not accounted to FIELDSIZE (and idem STRICTSIZE)
- is it acceptable that ALIGNLEFT & FILLZEROS does a left padding with dashes (instead of zeros) or is there a more intuitive naming for FILLZEROS (ALTFILL?)
- is there a suggestion for an intuitive operator to remove format qualifier e.g. (myFormat & ~FILLZEROS), where &~ is captured in a single operator e.g. (myFormat != FILLZEROS)
- is it acceptable to break backward compatibility regarding not suppressing the decimal point for PRECISION(0)
- is it acceptable to break backward compatibility for print(item, 0) and print(item, 1) // with item integer or equivalent type
  options 1) This implementation:            print(item, 0) => print(item) // implies DEC    print(item, 1) => print(item, DEC)
               2) Fully backward compatible: print(item, 0) => print((char) item)            print(item, 1) => print(item, DEC)
               3) compromise 1                      print(item, 0) => print(item) // implies DEC    print(item, 1) => print((char) item)
               4) Only reasonable thing :        print(item, 0) => print(item) // implies DEC    print(item, 1) => print("") // reserved

(for comments on PrintSequence, please wait for the PrintSequence design discussion thread - to be started soon)

Michael Jonker

[Federico Fissore]

Hi Michael

can you make a pull request of your code? This way:
1) we all know exactly what has changed
2) we can make the bot build your work so that giving it a spin it's
just a matter of downloading yet-another ide, instead of building it (in
essence: may involve not-so-technical people)
3) we all can comment on you code, right below it

Thank you in advance

Regards

Federico

Michael Jonker ha scrito il 03/03/2015 alle 23:42:

> Hello,
>
> The implementation of the Print enhancement is available for review.
> This thread is a follow up of the thread "Adding format control to the
> Print class methods";
>
> You can find the implementation in my github fork /MichaelJonker/Arduino
> in the branch review_FormattedPrint :
> i.e. https://github.com/MichaelJonker/Arduino/tree/review_FormattedPrint
>
> The relevant files in here are under hardware/arduino/avr/cores/arduino
> (location conforms to release 1.6)
> PrintFormat.h // defines the print format control classes (including
> the implementation of all methods)
> Print.h // defines the print interface (and inline
> implementations of trivial Print methods)
> Print.cpp // implementation of the non trivial Print methods
>
> In short, this code will allow the user to code things like:
>
> |

> Serial.println(3.141592,PRECISION(4)&FIELDSIZE(3));// 3.1416 ; note

> FIELDSIZE only concerns integer part, not the fraction
> Serial.println(66,HEX);// 42 ;
> Serial.println(22,RADIX(5)&FIELDSIZE(8)&ALIGNLEFT);// 42 ;

> Serial.println(34,OCT &FIELDSIZE(8)&FORCESIGN &FILLZEROS);// +00000042 ;

> note forced sign does not consume FIELDSIZE

> Serial.println(54,13);// 42 ; backward compatible base argument
> Serial.println(3.141592,3);// 3.142 ; backward compatible precision argument

> |
>
>
>
> There are some more handy features. These can be discovered from the
> sample code stored in the above mentioned github branch under
> examples/test_FormattedPrint
> testFormattedPrint.ino ino sketch which calls PrintDemo().
> PrintDemo.cpp demo showing the usage of the Print and
> PrintFormat classes.
>
> Notes:
> - The file PrintDemo.cpp contains instructions to compile and run the
> PrintDemo in standalone mode outside the Arduino environment (I used
> this under Cygwin for a fast development cycle)
> - The file PrintSequenceTeaser.cpp, also present, is a teaser (aka
> preview) of a PrintSequence class (still in the design stage).
> (However, please keep this thread focussed on the Print and PrintFormat
> classes and do not comment features related to the PrintSequence class
> in this thread.
> Discussion related to the PrintSequence class will be launched in
> different thread soon.)
>
> The Print and PrintFormat classes are functionally complete. I plan to
> work on optimisation (code reduction) and the implementation of a
> scientific print format.

> I have chosen for the print format syntax in the style: *OCT &
> FIELDSIZE(6) & FILLZEROS*.
> A syntax style like *OCT.withFieldSize(6).withFillZeros()* has not been

> provided (but could be added if this is what the community want).
>
>
> I have not implemented a PrintFormatter class for custom formatting.
> The primary reason is that one cannot define templates for virtual
> methods making a hypothetical abstract PrintFormatter class in first
> instance boring and inefficient (large vtable).
>
> Furthermore, such a class makes only sense for basic types, non basic
> types need to implement a Printable interface for them to become
> printable. Hence there is already full control over the format.
> With the latter in mind, one can easily see that custom formatting of
> basic types can be readily achieved through an adapter class:
>
> |
> // Using an adapter class IPAddressFormatAdapter
> classIPAddressFormatAdapter:publicPrintable
> {
> typedefunsignedcharIPAdress[4];
> IPAdressmyIPAddress;

> staticunsignedlong&_asLong(IPAdress&anIPAdress){return(unsignedlong&)anIPAdress;}//

> interpret an unsigned long as a unsigned char[4]
>
> public:
> IPAddressFormatAdapter(unsignedlonganIpAddress){_asLong(myIPAddress)=anIpAddress;}
>
> size_t printTo(Print&out)const
> {
> unsignedcharnc=0;

> for(inti=0;i<4;i++){if(i!=0)nc+=out.print('.');nc+=out.print(myIPAddress[3-i]);}//

> I'm an endian, I'm a little endian...
> returnnc;
> }
> };
>
> // the following hypothetical (because not available) method invocation:
> // Serial.print(0x21428418), myIpAdressFormatter);
> // may be implemented as:
> Serial.print(IPAddressFormatAdapter(0x21428418));
> |
>
>
>

> *In particular I would like your advise/comments on:*

> - is it acceptable that all PrintFormat constants are part of the
> namespace ArduinoPrint, and that this name space is automatically 'used'
> by including Print.h
> I would prefer a nested namespace like using Arduino::Print, but this
> change can be made later.
> - are the chosen PrintFormat constants acceptable? i.e.: BIN, OCT, DEC,
> HEX, RADIX(n), FILLZEROS, PRECISION(n), FORCESIGN, FIELDSIZE(n),
> STRICTSIZE, ALIGNLEFT

> - is the chosen PrintFormat mergeoperator& acceptable (e.g. HEX &

> --
> You received this message because you are subscribed to the Google
> Groups "Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to developers+...@arduino.cc
> <mailto:developers+...@arduino.cc>.

[Tom Igoe]

Also, as many example sketches as you can add will help to give users an idea of what can be done with it.

t.

> To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[David Mellis]

I haven't read over this in detail, but I think it's a bad idea to use bitwise-and (&) for this. We don't, as far as I know, require it in any other APIs and while it's useful for people to know, it doesn't seem like the kind of thing you should have to explain just so people can format their print statements.

--

[Rob Tillaart]

I would preferred a standalone formatter

double p = 3.141592;

format(buffer, p, PRECISION(4) & FIELDSIZE(3))

Serial.println(buffer);  

A separate formatter would 

- not change print at all,

- easier to debug as e.g. Serial.println(p); would be possible 

- allow the formatter to be used for other purposes like sending formatted data over I2C which does not implement print interface

just my 2 cents..

[Rob Tillaart]

good points

On Fri, Mar 6, 2015 at 9:11 AM, Peter Olson <pe...@peabo.com> wrote:

> On March 6, 2015 at 2:54 AM Rob Tillaart <rob.ti...@gmail.com> wrote:
>
>
> I would preferred a standalone formatter
> double p = 3.141592;
> format(buffer, p, PRECISION(4) & FIELDSIZE(3))
> Serial.println(buffer);
>
> A separate formatter would
> - not change print at all,
> - easier to debug as e.g. Serial.println(p); would be possible
> - allow the formatter to be used for other purposes like sending formatted
> data over I2C which does not implement print interface

Two issues:
  limited memory space for the buffer (which could be large) ...
    print as it now stands may need to block, but does not consume
    excess memory
  as written, the is no protection from buffer overrun (more likely
    to happen with larger strings than expected than with numerics)
    buffer overrun is quite a lot harder to debug than it is in
    other C/C++ environments due to the limitations of Arduino
    debugging; you can imagine in many cases, the sketch just
    becomes nonresponsive and you cannot tell why or where the
    damage occurred

Peter Olson

[Peter Olson]

> On March 6, 2015 at 2:54 AM Rob Tillaart <rob.ti...@gmail.com> wrote:
>
>

> I would preferred a standalone formatter
> double p = 3.141592;
> format(buffer, p, PRECISION(4) & FIELDSIZE(3))
> Serial.println(buffer);
>
> A separate formatter would
> - not change print at all,
> - easier to debug as e.g. Serial.println(p); would be possible
> - allow the formatter to be used for other purposes like sending formatted
> data over I2C which does not implement print interface

[Michael Jonker]

Hello,

@Federico :
I have put my changes in pull request #2743 from MichaelJonker:PrintNewFeatures

@Tom :
I have added one sketch with many examples fully contained in the .ino file. (which is a bit more evident and readable than the previous test sketch). The example can be found under [build/shared/]examples/11.Print/PrintNewFeaturesDemo

@all :
I have also packaged the new features demo in a zip file that contains a user library with the Print and PrintFormat classes as well as the demo sketch (slightly modified to include the user library). This zip (27K only and attached to this message) can be installed in your sketchbook and library environment without having to load a new IDE.

@David :
I think that one should not consider the & operator in e.g. HEX & FILLZEROS as a 'bitwise and' operator but as a PrintFormat merge operator, much in the same way as in c++ stdlib stream << 42 is not considered as a bit shift but as stream append operator. I choose this operator as I though that semantically speaking it was most suitable, in the above example it expresses that the merged format should have both HEX and FILLZEROS properties.

Does your reluctance against using this way of merging PrintFormats concern operators in general or the & operator in particular?

Alternative options would be:

HEX & FILLZEROS
HEX | FILLZEROS     // same category as operator|, semantically weaker than operator&
HEX + FILLZEROS     // good alternative, semantically weaker
HEX , FILLZEROS     // very confusing when used in parameter lists, I suspect that this will be very error prone.
HEX.withFillZeros() // leads to implementation complications and (in my humble opinion) is not very Arduino like.

@Rob
I did considered an implementation which would first produce a string like:

template <typename TYPE_T> String& Format(TYPE_T item, PrintFormat aFormat)
template <typename TYPE_T> size_t  print (TYPE_T item, PrintFormat aFormat) { return print(Format(item, aFormat)); }

such implementation would allow what you proposed: Serial.print(Format(66, HEX & FORCESIGN));

However, this implies creating temporary string objects (possibly of varying length), which means extra complications and I am a bit wary to open the door for memory management in the Print related methods.

Furthermore the overhead of the PrintFormat parameter will not dramatically increase the footprint. All 'fixed' PrintFormat instances can be derived at compilation time and generates two bytes of data. The underlying print code which does the actual work is implemented quite efficiently. In my previous version that provided only a 'raw/binary' format specification, this underlying code had for some cases even a smaller footprint. I will work on the footprint optimization later.

@all:

have fun

Michael

[Collin Kidder]

On Mon, Mar 9, 2015 at 2:55 PM, Michael Jonker
<michael....@gmail.com> wrote:
>
> @David :
> I think that one should not consider the & operator in e.g. HEX & FILLZEROS
> as a 'bitwise and' operator but as a PrintFormat merge operator, much in the
> same way as in c++ stdlib stream << 42 is not considered as a bit shift but
> as stream append operator. I choose this operator as I though that
> semantically speaking it was most suitable, in the above example it
> expresses that the merged format should have both HEX and FILLZEROS
> properties.
> Does your reluctance against using this way of merging PrintFormats concern
> operators in general or the & operator in particular?
> Alternative options would be:
>
> HEX & FILLZEROS
> HEX | FILLZEROS // same category as operator|, semantically weaker than
> operator&
> HEX + FILLZEROS // good alternative, semantically weaker
> HEX , FILLZEROS // very confusing when used in parameter lists, I
> suspect that this will be very error prone.
> HEX.withFillZeros() // leads to implementation complications and (in my
> humble opinion) is not very Arduino like.
>
>

I would personally prefer the syntax: HEX | FILLZEROS because that is
what people who are used to embedded development are already used to.
Bitwise OR is extremely common for lumping together several different
parameters and so I would imagine most programmers are used to this
syntax. & actually confuses me slightly when reading because I'll read
it as bitwise AND which makes me think that it does something else
entirely (mask operation). So, it might just be my background but I'm
not a fan of using & as a way to specify multiple conditions.

-Collin

[David Mellis]

Many Arduino users are unlikely to encouter & or | in normal use. I'd argue, in fact, that this is one of the many benefits of the API: allowing reading and writing individual pins without needing to use bit manipulation operators. I'd suggest specifying these options without using & or |. I like the idea of simply allowing to be specified as additional parameters, in arbitrary order (to be or-ed together inside the functions), but there are probably other ways of providing this functionality without requiring users to learn new operators / syntax.

[Roger Clark]

IMHO. This is a totally non standard way of handling print formatting, which basically stems from the lack of a decent implementation of either printf or sprintf

The problem only exists on AVR's, and not on Due, and not on the myriad of 3rd party boards like PIC32, Teensy and STM32  - as AFIK they all have a full implementation of sprintf

What percentage of users will use this as opposed to using sprintf, dtostrf  or even writing their own custom formatting routines.

Do we really want to teach people that this is the normal way to format printing?

[William Westfield]

> This is a totally non standard way of handling print formatting

I have to say that I agree with Roger. Not that printf is wonderful, but at least it’s standard :-(
(Are there any other standard output formatting schemes, post-ForTran? You’d think that there should be…)

> which basically stems from the lack of a decent implementation of either printf or sprintf

Is the avr-libc printf so bad, other than having two versions for float/non-float?
(Could the preprocessor be made smart enough to pick the correct printf library?)

BillW/WestfW

[Andrew Kroll]

LISP has a pretty-print idea, it dates back to the 89 standard.
Perhaps some concepts can be gleaned from it?

https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node253.html

--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

--

Visit my github for awesome Arduino code @ https://github.com/xxxajk

[Peter Olson]

> On March 12, 2015 at 3:08 AM Andrew Kroll <xxx...@gmail.com> wrote:
>
> LISP has a pretty-print idea, it dates back to the 89 standard.
> Perhaps some concepts can be gleaned from it?
>
> https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node253.html

I am pretty happy with the current print/println functions, except that I
dislike having to put print(' ') commands all over, to separate fields in an
output line. More typing when I am editing the source.

I am mostly programming Dues, so the code size is not an issue (the largest of
my projects takes 9% of available code space :-)

I have a showExpr(expr) macro which evaluates as print("expr = ");
println(value); where value is the value of an expression which shows in the
first print command and expr is the literal expression being shown (e.g.,
loopTime / 1000 or numloops - 1)), e.g.:

loopTime / 1000 = 0.001027

These prints are actually serobj.print and I have a mechanism for enabling or
disabling print levels at run time to avoid excess upload traffic to the host.
It's experimental at the moment.

This is partly why I like the idea of an external serial monitor. My Python
monitors work with the Arduino codes to display a subset of information on the
screen but save everything to a file on the host for debugging.

Peter Olson

[Peter Olson]

> On March 12, 2015 at 2:55 AM William Westfield <wes...@mac.com> wrote:

> Is the avr-libc printf so bad, other than having two versions for
> float/non-float?
> (Could the preprocessor be made smart enough to pick the correct printf
> library?)

Unfortunately no, because the format argument can be passed as a string variable
whose contents is not knowable at compile time.

Peter Olson

[Adrian Godwin]

You can't tell for sure what the format string contains, but if there are no FP objects declared at all, there won't be any to print. This doesn't satisfy the condition where FP variables are used but only integers printed, but it helps.

[Andrew Kroll]

Actually, you can tell the type, when you are using an overloaded method you will know the type.

Another way to get around the type problem would be to use templates.

Finally, the last way to get around it all would be to use some macro tricks and generate a format string for the normal C library functions.

I say keep it as simple as possible. This sort of thing is exactly why some print formatting ideas die off, and others remain.

[elizarov]

Hello,

I really like the idea. However, I think that implementation can be improved. I'll confess that I have my own formatting library for Arduino. It actually lets me avoid floats altogether. I write all my code using "FixNum" where the number of digits after decimal point is known at compile time. So, there is a FixNum library that keeps track of all the conversions (like knows to multiply mantissa by 10 when I assign fixnum with precision of 1 digit to a fixnum with precision of 2 digits). But that is not relevant, since FixNums, in a way I use them, still require a certain care and discipline and I have not figured out yet how to make it simple enough for a general Arduino audience. You can still take a look, though, if you want: https://github.com/elizarov/haworkslibs/blob/master/FixNum/FixNum.h

Back to formatting. There is an explosion of Print class in this proposal. Actually, I don't like existing Arduino's "Printable" with its virtual "printTo" method, either. The way I do it, it that all my "printable" classes have a "format()" method that returns a temporary char buffer class that has a defined conversion to "char *". Take, for example, my DateTime class for RTC clock. I print the current date and time like this:

  Serial.println(dt.format());

The way I would format a number that is returned by some "getMyValue()" method is:

  Serial.print(getMyValue().format()); 

Given the fact, that getMyValue() returns an appropriate FixNum object (which just a wrapper upon an appropriate integral type without any extra runtime overhead) . This way I don't even have to keep track of how many digits of precision "getMyValue()" returns. It will be formatted with a many decimal digits as getMyValue had intended to provide. However, I can still provide precision and formatting options explicitly as an argument to "format()" method.  However, the way I do this is not as elegant as in your proposal.

In order to make it all fit together, Arduino Print class shall define just one new method:

  template<typename T> inline size_t print(T value) { return print(value.format()); }

This way "Print" class can be kept small and nice as it is now, while the formatting of all objects is completely delegated to library writers. 

How do you define the library for integral and float formatting, then? It is as easy as defining a separate hierarchy of PrintFormat (just as you propose) and _FormattedValue_ classes that encapsulate a pair of typed value and format (their typical use will be inlined by compiler), With an appropriate "&" operations defined between the types, printing can be as easy as writing the following code (I'm expanding your examples):

  Serial.println(3.141592 & PRECISION(4) & FIELDSIZE(3));

  Serial.println(66 & HEX);                                    

  Serial.println(22 & RADIX(5) & FIELDSIZE(8) & ALIGNLEFT);

So you have the same ease of use for the end user, no virtual invocations what-so-ever would be needed at run time, but still fully extensible by any library writer to support any other types or formatting options. The only downside, is that for type-conversion purposes we'll have to ditch the "print(value, format)" pattern of usage and switch to "print(value & format)" pattern instead. I cannot figure out how to make it work with comma, so that it is all nice, short, extensible and backwards compatible.

Note, that end users will never have to deal with templates. They will be used in Arduino's Print class only for benefits of library writers. 

[Andrew Kroll]

On Fri, Mar 13, 2015 at 6:54 AM, elizarov <eliz...@gmail.com> wrote:

Hello,

Hi!

<SNIP!>

The only downside, is that for type-conversion purposes we'll have to ditch the "print(value, format)" pattern of usage and switch to "print(value & format)" pattern instead. I cannot figure out how to make it work with comma, so that it is all nice, short, extensible and backwards compatible.

I would do a pile of overwritten methods (or a template) that would do something like this:

void print(uint8_t value, int8_t format=-1) {

        if(format >=0) print(value & format);

        else print(value);

}

 

[Michael Jonker]

@printf -- (or here we go again)
If printf were available in the IDE for the UNO, (and with a low footprint), I would not discuss this.
I needed a lightweight formatting extension, I implemented it, then I proposed to share it (see "Adding format control to the Print class methods" https://groups.google.com/a/arduino.cc/forum/?fromgroups#!topic/developers/7KpdLDgFsO0).

The initial implementation provided only an efficient, low footprint print formatting extension, with a 'primitive' API. Following the discussions, I implemented a friendly Arduino-like API.

The alternative printf has already been discussed at length in the above mentioned discussion. Not only is printf very Arduino-_unlike_, for me printf is 'HAS BEEN'. Although I use it a lot, it is not the thing I would teach my kids.

Why cursing printf? printf needs run time interpretation of the format string, which implies: a) type unsafe, e.g. printf("%s",3.14); and b) a large footprint, all bells and whistles of printf are loaded with the first usage of printf, whether you need them or not (footprint is of concern on the UNO).
(I know that gcc allows format checking with the __FORMAT__ Function-Attribute, but that is gcc specific and does not allow to check run time format strings... which are needed if you want to control certain formatting attributes (Radix, Precision, FieldSize) at run time. Run time format is anyway a pain with printf format strings - and I should have added this as point C) to my curse list.

Any alternatives?
There is an alternative available today that can be downloaded from PR #2743 https://github.com/arduino/Arduino/pull/2743
Arduino users already know this:
Serial.print(66, HEX);

with the new implementation they can do (amongst others)
Serial.print(66, HEX + FILLZEROS + FIELDSIZE(6) ); // *)
*) note in the current PR #2743 it is: HEX & FILLZEROS & ... (I will change & to + in the final version for various reasons).

@Andrew Krol.

The lisp inspired pretty print alternative ... based on 'visual compact format strings'
an example from the reference: (format t "~:<~W~^ ~:<~@{~:<~@{~W~^ ~_~}~:>~^ ~:_~}~:>~1I~ ~@{~^ ~_~W~}~:>" list)
wow that is pretty, I challenge you to convince Tom Igoe.

Ok, I guess you refer to the model of the format delegation. This is technical implementation details, not an inspiration for a better API.

The best inspiration for a more conventional API is provided by the C++ stream append operator. This was already addressed in the above mentions discussion. I have an implementation of c++ stream style API as an independent supplement lined up. This API provides a syntax like:
out <<"reg=" <<reg <<" (0x" << HEX+FILLZERO+FIELDSIZE(8) << reg <<")" <<NEWLINE;
When the PrintFormat discussion is settled, I will propose this c++ stream style API as an independent supplement.

Merge operator
@David
The comma as a merge operator is confusing. This leads to a syntax like:
myDefaultFormat = FIELDSIZE(8), FILLZEROS;
- The assignment syntax is not natural, how do we explain this to the novice user ??
- The syntax print(item, FORMAT1, FORMAT2, FORMAT3) compromises future extensions of print()
- It is tricky to implement, one has to overwrite the comma operator (for the assignment), deal with possible conflicts, etc.

Bitwise logical operators... & |
The operators & and | are only bitwise logical operators in the context of int. The arguments in the previous postings are quite diverging, David states that the Arduino users are ignorant of bitwise operators, Collin implies the opposite. In reality we will have both groups of users, but with good examples both groups will learn that the PrintFormat merge operator merges PrintFormats. They will learn that the meaning of operator symbols will depend on their context. They will appreciate that there can be several solutions to a similar problem and that the specific solution chosen usually suits the environment best.

Is it only a problem for those who got locked into old habits?

I went over the discussion "Adding format control to the Print class methods".
There is a majority of people in favour of + (amongst the merge operators). It was first suggested by Paul Stoffregen.
Due to the operator precedence it also simplifies the syntax when used together with the stream append operator.
==> I will adopt the choice + operator.

to be continued

[Andrew Kroll]

Exactly, It's about gathering ideas.

There are lots out there, LISP was an example, there's also various formatters for Python that could fit as well (conceptually). Python also uses the '+' operator. I like this idea actually.

What I am suggesting is that perhaps if we glean the best from everything else as a mix, we can have something that is not only simple to use, but also light weight. Instead of a format string per-se, in some cases it may make more sense to provide formatting in-line. However this is not always the case, since use of too many can lead to a jumbled mess of a line.

--

You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[Michael Jonker]

On Friday, March 13, 2015 at 11:54:08 AM UTC+1, elizarov wrote:

Hello,

I really like the idea. However, I think that implementation can be improved. I'll confess that I have my own formatting library for Arduino. It actually lets me avoid floats altogether. I write all my code using "FixNum" <snip>

Back to formatting. There is an explosion of Print class in this proposal. Actually, I don't like existing Arduino's "Printable" with its virtual "printTo" method, either. The way I do it, it that all my "printable" classes have a "format()" method that returns a temporary char buffer class that has a defined conversion to "char *". Take, for example, my DateTime class for RTC clock. I print the current date and time like this:

  Serial.println(dt.format());

The way I would format a number that is returned by some "getMyValue()" method is:

  Serial.print(getMyValue().format()); 

Given the fact, that getMyValue() returns an appropriate FixNum object (which just a wrapper upon an appropriate integral type without any extra runtime overhead) . This way I don't even have to keep track of how many digits of precision "getMyValue()" returns. It will be formatted with a many decimal digits as getMyValue had intended to provide. However, I can still provide precision and formatting options explicitly as an argument to "format()" method.  However, the way I do this is not as elegant as in your proposal.

In order to make it all fit together, Arduino Print class shall define just one new method:

  template<typename T> inline size_t print(T value) { return print(value.format()); }

This way "Print" class can be kept small and nice as it is now, while the formatting of all objects is completely delegated to library writers. 

How do you define the library for integral and float formatting, then? It is as easy as defining a separate hierarchy of PrintFormat (just as you propose) and _FormattedValue_ classes that encapsulate a pair of typed value and format (their typical use will be inlined by compiler), With an appropriate "&" operations defined between the types, printing can be as easy as writing the following code (I'm expanding your examples):

  Serial.println(3.141592 & PRECISION(4) & FIELDSIZE(3));

  Serial.println(66 & HEX);                                    

  Serial.println(22 & RADIX(5) & FIELDSIZE(8) & ALIGNLEFT);

So you have the same ease of use for the end user, no virtual invocations what-so-ever would be needed at run time, but still fully extensible by any library writer to support any other types or formatting options. The only downside, is that for type-conversion purposes we'll have to ditch the "print(value, format)" pattern of usage and switch to "print(value & format)" pattern instead. I cannot figure out how to make it work with comma, so that it is all nice, short, extensible and backwards compatible.

Note, that end users will never have to deal with templates. They will be used in Arduino's Print class only for benefits of library writers.

Hi Elizarov

I read your reply with interest and I understand your concept. (I also like your FixNum concept -- it must have been fun to implement, I wonder about rounding, additions and how to trade-off between overflow and precision -- but this is off topic).

A similar concept with the objectives to separate the formatting from the Print class was already addressed in the discussion with by Matthijs Kooiman (27 Jan on the above mentioned discussion). He proposed:

template<typename T, typename F>
        inline size_t print(T value, F format) { return format.printTo(*this, value))};

This makes formatting generic and it permits to delegate formatting to PrintFormat classes, but without the need of passing a temporary string. I intend to go this way in a future version although it will not change much in the footprint: As I now see it, when delegating formatting to PrintFormat, the implementation of the extended formatting primitive will still be the same. I am not worried about slightly extending the current formatting capability of the Print implementation, the overhead is small.

Right now I have it all implemented, but my biggest problem is to have this syntax accepted:

Serial.print(value, aPrintFormat);

which should be trivial as we know already

Serial.print(66, HEX);

It gets more complicated with the PrintFormat merge operator, like either HEX + FILLZEROS or HEX & FILLZEROS.

I think it will be an even bigger hurdle to convince the community (including me) to merge value and PrintFormat, like: 66 + HEX + FILLZEROS

Once PrintFormat and a merge operator is available anyone can add privately (or propose) his operator that merges a value and PrintFormat into a printable (or string) giving what you were asking for.

BTW I have the intention to replace the printable interface itself with a template specification that does not need the virtual method. Also detailed in the mail by Matthijs.

template<typename T> inline size_t print(T item) { return item.printTo(*this): }

(I have to resolve some conflicting method signatures, further for this to work in general with embedded classes (i.e. no external linkage) one needs c++11 support, and the -std=C++11 option is not part of the Arduino compile environment).

Related to virtual methods, please note that print has itself already a virtual method write(), I suspect that this upside down hierarchy of print can be considered as a historical mistake...

Otherwise, I do not see much difference in implementation overhead between my proposal and your proposal. I will have to do some implementation studies to see this in detail. You mention an explosion of the print class, but that is in the API, not the implementation (and most of this will get resolved at compile time). The increase in footprint is probably rather modest (based on studies of my previous implementation, which I still have to repeat for the current implementation). The API header files are already using templates a lot, I intend to shorten it further once this implementation is accepted.

I concentrated on getting formatting of numbers in first. It is implemented and can be tested, but now I spend most my time on trying to get a consensus.
This is where I need support.

Cheers
Michael

 

[Andrew Kroll]

You could always make it a totally new class, and leave print alone :-)

--

You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[William Westfield]

> I needed a lightweight formatting extension, I implemented it

Is your extension that much more light-weight than printf()? My chief concern is that we’d eventually get “weight” and complexity similar to printf(), without being compatible with … anything (and possibly leaving out “necessary” formats that had already been implemented in printf())

> Is it only a problem for those who got locked into old habits?

Maybe. When I worked on adding formatted output to a Pascal compiler, I think I used Fortran-style format strings, because they were a "well-known and complete” example. :-) (I don’t recall whether that code ever saw light of day, but even if so it’s long gone now.)

BillW

[David H. Lynch Jr.]

Microsoft had an implimentation of printf in x86 assembler that did nearly everything except floating point, in about 690 bytes of code.
There is a fairly small (300 lines of sparse C) implimentation of printf in the linux kernel.

[Andrew Kroll]

I think i can locate my printf that was possibly smaller, its for z80, but is plain ansi C.

[bob cousins]

On Sunday, March 15, 2015 at 2:44:57 PM UTC, Michael Jonker wrote:

The alternative printf has already been discussed at length in the above mentioned discussion. Not only is printf very Arduino-_unlike_, for me printf is 'HAS BEEN'. Although I use it a lot, it is not the thing I would teach my kids.

I would say the same about 8 bit AVRs with tiny amounts of memory...they are not the future. Recently announced Cortex-M7, 300 MHz, 1MB Flash, 384 KB RAM...

It does sound like a smaller version of printf for AVR is what is needed, rather than re-inventing formatted IO in a non-standard way. printf has flaws, but it is compact in terms of user code lines if not library footprint, and printf is so well known and used.

I have no objection to this extension being included, assuming it takes up no overhead if not used, but I probably would never use it.

[William Westfield]

> Microsoft had an implimentation of printf in x86 assembler that did nearly everything except floating point, in about 690 bytes of code.

Um. The libprintf_min.o library in avr-libc (which is the default, non-floating-point version) is 689 bytes.
If I understand things correctly, that implements vfprintf() the core routine that does the formatting and argument parsing and so on, so there are small “wrapper” functions like printf() itself (40 bytes.) The wrapper functions get garbage-collected, but vfprintf() is a single largeish function.
The floating point vfprintf() is 1790 bytes.

By comparison, Print.cpp.o from the 1.6.0 IDE is 1620 bytes for everything. That gets rather aggressively garbage-collected in sketches that don’t use everything, so that ASCIITable, for example, has about 600 bytes of Print:: functions.

That doesn’t leave a lot of room for additional features, before even a minimal Print-based sketched becomes bigger than one using printf() that has even more features. And is standard, regardless of ugliness.

ARM is more complicated; I’m not sure exactly which library is used, and I doubt whether it has been nearly as well optimized. _vfprintf() in lib/thumb/libc.a is more that 5k.

BillW
WestfW

[William Westfield]

> That doesn’t leave a lot of room for additional features

That said, I made a version of ASCIITable that uses printf() instead of print::, and the printf version is about 900 bytes bigger. In addition to vfprintf(), it sucks in several other support functions, including a rather substantial fputc() and some ultoa variant (and it still doesn’t have binary!):

00000730 00000028 T printf
00000758 000003a4 T vfprintf
00000afc 00000016 T strnlen_P
00000b12 00000016 T strnlen
00000b28 00000064 T fputc
00000b8c 000000bc T __ultoa_invert
00000c48 00000038 T __prologue_saves__
00000c80 00000036 T __epilogue_restores__

BillW

[David Mellis]

Another thing to keep in mind is that printf() can be confusing and error prone for people that haven't seen it before. And much of the intended audience for Arduino is people who don't already know C, and therefore don't already know printf(). 

[Andrew Kroll]

Perhaps they just need to read a tiny bit more. I think they may also benefit more if someone just gave them a PDF to print that has all the printf format string details, written in a way that a noob can understand... e.g. as soon as the idea clicks, have them write the description.

[Tom Igoe]

No. RTFM is not the answer.

t.

[Andrew Kroll]

So then if RTFM is not the answer, why is there a "language reference page" on arduino.cc for people to read?? hmmmm?

Lets be realistic here, even if you make an app that craps out formatting strings for some one, they'll have to read or watch a video in order to use that app.

[Tom Igoe]

Sorry, let me be more clear (and apologies to long-time users of this list who’ve heard me say this before):

There is a difference between “RTFM” and “Here’s a getting started guide that may help.” One says “I haven’t got time for you, not even enough time to type out ‘read the fucking manual’ ” and the other says “Here let me help you, I’ve answered this before, and maybe that answer will work for you too.” I’m not calling for a no-documentation solution, I’m calling for a less terse solution that assumes the person is willing to read but might have a different set of background assumptions. Not all Arduino users want to be programmers. But they do want to be able to use programming. 

Another way of putting it:  we’re all users of language, but most of us here are not professional writers. Yet we all know someone who will slap us upside the head with Strunk & White’s “The Elements of Style” every time we dangle a participle or split an infinitive. It doesn’t help, and it’s discouraging. it’s equally discouraging when we slap beginning programmers upside the head with Kernighan & Ritchie every time they ask what the & is for, or with Horowitz & Hill every time they use the wrong resistor.

Instead, what I’m calling for is a critical look both at the beginner’s understanding and expectations, and at the strengths and limits of our tools. C/C++ is powerful, but it’s also terse in places, and opaque in places. Rather than assume that printf is good enough for everyone, I would rather look at its shortcomings and find a way to make it less opaque so as to be usable by a wider audience. Then when we have a solution, write a clear and approachable explanation of its use, written for a general audience.

To be clear, I am NOT asking for anyone here to help beginners if they don’t want to, but as long as we’re thinking about extending the API, I am asking that we do it in a way that improves its legibility to a general audience.

Tom

[Tom Igoe]

Although, if you’ve seen a write-up of printf that’s clear and approachable for a general audience, please post a link to it here. I’m always on the lookout for ways to make the harder parts clearer.

[Andrew Kroll]

https://www.youtube.com/watch?v=w2LqUtmM10M
https://www.youtube.com/watch?v=zPopGTKhm-s

and many more on youtube...

[Tom Igoe]

The former I can’t  comment on, as my Spanish is not good enough, sorry. As for the second one, the style is very much aimed at a technical audience. There are a number of terms used, even in the first thirty seconds, that require a background in computer science, that would turn off most of the folks we’re trying to reach. Compare these to the tone of, for example, Massimo’s beginning Arduino videos and you’ll see a marked difference. It’s these latter that we’re aiming for. https://www.youtube.com/watch?v=7JoQvT1oEdA You could also look at the starter kit documentation itself and how it explains concepts.

I’m happy to provide counter-examples and explain my position further, but only if you’re actually interested. If not, let me know, and I won’t waste your time. 

[Andrew Kroll]

Those were meant as examples.

[Tom Igoe]

I understand. But perhaps I wasn't clear in what I'm looking for, sorry. I'm looking for examples that are aimed at a general audience, not a technical audience. The difference in pedagogical approach in Massimo's video compared to the ones you posted is, to me, the core of our approach to Arduino. Sorry I wasn't clearer earlier. If the difference isn't clear, let me know and I'll try to be clearer, provide more examples, as needed.

sent on the go. please excuse brevity and mistaken auto corrections

[Jack Rickard]

I think you guys are walking a careful line here.  

My initial impression with Arduino some years back was that it was a weeny version of C for artists trying to make LED's light up.  I was delighted to find C++ lying beneath with full class structures, object oriented, inheritance and even syntactic sugar I was comfortable with.  Really most of C++ without STL. No mention of MOST of it in the documentation.  

A LOT of that was undocumented in the Arduino stuff.  But most of what I tried that wasn't STL seemed to work just fine and I gradually became a big fan.

I think you have a bit of a fork in the road here.  I would like to see printf() in.  And I do not think there is any obligation to document it, clutter the Arduino docs with explanations of it, or push it on new users.  It could just be there if you know how to use it, and like MANY other features that are never mentioned, it will work just fine.

The other fork is to do an Arduino print format.  printf is actually terribly awkward in many ways particularly for new users.  Here's an opportunity to do a clean sheet on this frequently needed utility.  But then that has to be thoroughly described and documented as it just isn't standard.

For me, I'd just add printf and never mention it to a soul.  Nobody doing robotics and lighting LED's is going to need it.  Those who know how to use printf will inevitably try it (as I have) and be delighted to find it (which I wasn't).  I just don't think Arduino is really best of type when it comes to printouts.  But that's not really what it was designed to do.  Making it into Wordstar probably won't work.  

The other little tiny gripe is I sure wish the terminal at least did the basics, where a carriage return returns to the start of the line and a newline goes to the next line and maybe a form feed or top of form, so we could do USB displays that didn't trigger epileptic fits with all the scrolling.  Yes, yes, you can use a terminal.  But screen is kind of icky for new users and hyperterm isn't even included in Windoze anymore.  We often just point the end users to the Arduino IDE to configure things over teh built in terminal program.  But it always has to scroll like crazy anytime we update a value.

Jack Rickard

--------------------------------

http://EVTV.me

---------------------------------