Serial.room(), or "ready" or "availableWrite" or "TxBytesFree" or....

[Paul Stoffregen]

We've talked about this API many times, always with a consensus it makes sense.  Recently, I've had more users request a way to avoid blocking in Serial.write().  I'm implementing it now on Teensy, and I hope someday it'll be implemented in Arduino's core library.

My hope is we can agree on the name for this function, so when/if it's ever implemented in Arduino's core library, we'll end up with the same name.  Long-term, I'd like to stay as compatible with the Arduino API as possible.  Yesterday I committed code with "room", but since it's only 1 day old, changing is still pretty easy.

Previously, I had preferred Serial.ready().  This forum thread swayed me to Serial.room().

http://forum.arduino.cc/index.php/topic,165574.0.html

Really, I don't care what the name end up being.  Feedback I've heard from several people is a design for a longer, more descriptive name.  Suggestions have included Serial.availableWrite(), Serial.writeAvailable(), Serial.readyWrite(), Serial.TxAvailable(), and other variations.  Any of these names is fine.  Agreeing on a name, so all boards implementing this remain compatible, seems like a good idea.


Regarding the feature itself, the last time I brought it up was item #3 on this API wish list.

https://groups.google.com/a/arduino.cc/d/msg/developers/PcOW-MCuDHc/KxLcOKLlNhoJ

There are at least 2 pending requests on the issue tracker:

https://github.com/arduino/Arduino/issues/672

https://github.com/arduino/Arduino/issues/1408

The implementation is really very simple.

[Matthijs Kooijman]

Hey Paul,

I like room() - it's nice, simple and I think also clear.

One possible complication is that it doesn't point out that you're
talking about room in the TX buffer (free room in the RX buffer could
also be relevant), but since you'll usually want to know TX room, having
a room() for TX and readRoom() for RX is probably ok - but I don't think
the need for this is big enough to add it now.

Regarding using writeAvailable or TxAvailabe - I don't like these. They
would refer to "available space", but the existing available() method
refers to "available bytes", which is completely the inverse and IMHO
confusing.

Regarding using ready or readyWrite - I'm not sure what the meaning of
these is? How much space is ready to be written? Sounds confusing to me.

So, +1 for room() for me.


Gr.

Matthijs

[Tom Igoe]

HI Paul,

For me, room() doesn’t tell me anything and is awkward. In fact, I couldn’t tell a lot from your email, either, though the forum thread clarified for me what the function was about. If this is truly a parallel command to Serial.available() but on the transmit side, then writeAvaliable(), txAvailable() or outputAvailable() make more sense.   In general, nouns don’t make good command names. Verbs, adjectives, and adverbs are better. There are likely exceptions, but I don’t think room() is a good one. -1 from me, and +1 on writeAvailable() or ready().

--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[Paul Stoffregen]

Thanks Tom.  That's excellent.

Really, I'm indifferent regarding the name.  I really only want to give users an API to avoid blocking the rest of their sketch from running.

I guess the name is really Cristian's call, as well as whether Arduino will someday implement this.  I'm just hoping a decision can be made.  I'll happily update my code to whatever name you, Chistian and the rest of the Arduino Team decide.

Or in the absence of a decision, I think I'll change it to ready(), only because the other Serial functions are single words.

[Knight, Dave]

To me this seems to be much ado about something that is handled transparently in most C/C++ libraries & runtime environments using larger/dynamic buffering of some sort. 

I believe that serial i/o buffering can and should be handled better in the core within the context of standard C/C++ i/o calls because the net result of moving control of serial i/o buffering to the application level is likely to be overly complex sketches that will be very difficult to debug because the problems result from asynchronous external events.

Just my 2 cents from 50 years of operating system internals work, 30 years of which in C & C++.

[Jimmy He]

I'd agree with Dave on the buffering issue. Unlike receiving data, a user doesn't necessarily need to check whether he is ready to transmit before actually doing so. Having a deliberate Serial.room() function is weird.

Instead I would also suggest having a dynamic buffer for tx within the core - that only reserves a small amount of memory space when idle, and having a smaller handler to keep track on the status of TX usage, and to decide whether more memory is needed for the transmit data to queue up a larger cache. This doesn't seem overly complicated both at the level of developers and users.

[Adrian McEwen]

What happens when your dynamic buffer wants to grow in size but the system is out-of-memory? 

I agree that such a system is a much better solution for systems with lots of memory, but as an embedded developer I go out of my way to have as statically sized memory usage as possible.  It's an age-old tension point in the Arduino environment, but at least so far I can (mostly) avoid it if I choose.

Which class is this proposed for?  Serial or Print?  It feels like it makes more sense to live in Print, although obviously that'll have more knock-on effect for other libraries (e.g. Ethernet, SD card, etc.)

Cheers,

Adrian.

[Knight, Dave]

When "the system is out of memory", results become unpredictable, no different from any sketch that causes a heap/stack collision. 

As long as all malloc'd buffers are freed immediately after use (this is not a good place to use realloc to "grow" a permanent buffer), this should not cause a problem unless a sketch is already running very close to a heap/stack overrun,  AND writes a lot of stuff to Serial, e.g. big log msgs, very frequently.  In that case, blocking a write (on some "low memory" threshold) would be appropriate to prevent a collision, and easily implemented in the core code.

The core implementation of the String class already uses malloc'd memory (via realloc()), so there is a precedent for dynamic allocation in the core.

FWIW, it would be really helpful (to sketch programmers) if "the system" could emit some kind of error message (to Serial?) when there is a heap/stack collision, or maybe even a warning when such is imminent (like maybe when there's only 100 bytes left between the heap & stack).  At present, the only indication of a collision seems to be when the sketch produces incorrect results, or simply stops running, due to some data being corrupted.

Conceivably, such an error message could also help to debug sketches that make extensive use of the String class and behave unpredictably.

[Victor Aprea]

This thread is dangerously close to becoming derailed / hijacked. The question at hand is *not* how to make send non-blocking for all sketch authors. It's just how to expose an API that *allows* a user to avoid blocking due to buffer congestion by intentionally deferring a transmission. With all due respect, the Arduino core is not an operating system and shouldn't be expect to act like one. 

For reference, I'm pretty sure the code in HardwareSerial that eventually leads to blocking is this:

size_t HardwareSerial::write(uint8_t c)
{
  int i = (_tx_buffer->head + 1) % SERIAL_BUFFER_SIZE;

  // If the output buffer is full, there's nothing for it other than to 
  // wait for the interrupt handler to empty it a bit
  // ???: return 0 here instead?
  while (i == _tx_buffer->tail)
    ;

The sketch author simply wants a way to introspectively determine whether they will be impacted by that "buffer is full" wait loop. Since they might reach this line using a call to Serial.print that emits multiple bytes (e.g. with a string argument), they would really like to know how many bytes of memory are available in the transmit buffer so that they can either send a chunk of the string now or wait until there is enough space for the full string in a single call later. 

I think "Serial.room()" isn't a great expression of this intent. Serial.transmitBufferBytesAvailable() seem to me like clearer expressions of author's intent. The reality is that most users will not use the function and will be content to be blocked. Those that do require it will be happy to find it exists at all. As such, I would err on the side of a "too verbose" method name, which I expect would be the main criticism.

Kind Regards,

Vic

[Matthijs Kooijman]

Hi Tom & Paul,

> Tom wrote:
> >For me, room() doesn't tell me anything and is awkward. In fact, I
> >couldn't tell a lot from your email, either, though the forum
> >thread clarified for me what the function was about. If this is
> >truly a parallel command to Serial.available() but on the transmit
> >side, then writeAvaliable(), txAvailable() or outputAvailable()
> >make more sense. In general, nouns don't make good command
> >names. Verbs, adjectives, and adverbs are better. There are likely
> >exceptions, but I don't think room() is a good one. -1 from me,
> >and +1 on writeAvailable() or ready().

Hmm, it seems Tom and I have totally inverted opinions on this:

MK> I like room() - it's nice, simple and I think also clear.

I don't mind a more verbose or explicit version, though.

However, I don't think this new function is a really parallel command to
available(). I wrote before:

MK> Regarding using writeAvailable or TxAvailabe - I don't like these. They
MK> would refer to "available space", but the existing available() method
MK> refers to "available bytes", which is completely the inverse and IMHO
MK> confusing.
So, the adjective part is the same, but the noun part (used/bytes vs
space/room) is completey inverted, making an IMHO big difference. Does
that make sense?

MK> Regarding using ready or readyWrite - I'm not sure what the meaning of
MK> these is? How much space is ready to be written? Sounds confusing to me.
Paul, care to respond to this? I _really_ don't know how "ready" is to
be interpreted, so I'm probably just missing the point.

Gr.

Matthijs

[Victor Aprea]

Re-reading this post, I think my first paragraph might have came across as rude and that was not my intent, so I just wanted to apologize. It wasn't my place to moderate discussion.

Cheers,

Vic

[Paul Stoffregen]

On 06/30/2014 09:03 AM, Matthijs Kooijman wrote:
> MK> Regarding using ready or readyWrite - I'm not sure what the
> meaning of MK> these is? How much space is ready to be written? Sounds
> confusing to me. Paul, care to respond to this?

Sure, I'm happy to explain.

But first, I want to be clear, I only meant to ask about naming this
function, for the sake of future compatibility, if Arduino even
implements it. I never intended to discuss memory allocation schemes or
design philosophy of embedded systems. I suppose such escalating scope
is inevitable on this mail list, but my intention was only to talk about
the name.

Your point is good, the functionality needs to be well understood to
choose a name.

Serial.room() is meant to report the number of bytes that may be written
without significant delay. Really, it's that simple.

Often sketches are designed to poll multiple inputs, such as reading
pushbuttons with digitalRead() or sensors with analogRead(). Many
sketches follow the example 02.Digital > BlinkWithoutDelay approach.
Many Arduino sketches have low performance requirements, where brief
delays are of little consequence, but others require lower latency.

Serial.write() delay causes terrible problems in timing sensitive
projects, particularly when controlling DC motors. Today, this leads to
2 unfortunate results. #1: novice users have a frustrating experience
where solutions offered on the forums involve estimating data rates, and
#2: advanced users build complex code that bypasses Serial and directly
manipulates the hardware, typically with poor compatibility for anything
other than Arduino Uno. LabView LIFA and DmxSerial are prominent
examples. My main hope is offering an API that allows sketches and
libraries to avoid ugly, direct hardware access when all they need is
this simple capability.

No matter how Serial.write() buffers data, there will *always* be a
limit. Serial.room() is meant to report the amount of data that can be
written without suffering extra delay.

Again, my hope is only to choose the name, in the interest of future
compatibility if/when Arduino might implement this function. Obviously
there is little consensus on implementation, judging from this
conversation. Whether we can all merely agree on a name is a good question?

[Tom Igoe]

Thanks Paul,

Given this, how about transmitBufferSize()? Then you get a construction like this:

if (Serial.transmitBufferSize() < TX_BUFFER_MAX ) { // I’m forgetting the exact constant name at the moment, but you get the drift, I hope
doSomething()l
}

And hey, I just found the noun example I couldn’t find earlier. Somehow size() or length() feel more straightforward than room().


t.

[Matthijs Kooijman]

Hey Paul,

thanks for your reply, but you didn't answer my actual question :-) I
understand what the function is supposed to do and I agree completely
about focusing on the name - redesigning serial buffering is a worthwile
discussion, but that was not the scope of this thread and, as you
mentioned, even when someone would implement dynamic memory buffering -
you'd still want to have a room() function even if it would just always
return INT_MAX or something.

So, my actual question was, why do you think the name "ready" is a good
name? I can't see how the name relates to the functionality. "room"
refers to "how much room is available in the buffer", but what does
"ready" refer to?

Gr.

Matthijs

[Paul Stoffregen]

Opps, sorry. I didn't intent to dodge your question.

> So, my actual question was, why do you think the name "ready" is a good
> name?

Honestly, I do not have a strong opinion about the name.

Obviously I was swayed to change from "ready" to "room" by conversation
on that Arduino forum thread. Before "ready", I had written earlier
proposals with other names.

Tom's point about verbs, adjectives, and adverbs seems compelling, so
I'm swayed back to "ready". I'm fickle! ;-)

> I can't see how the name relates to the functionality. "room"
> refers to "how much room is available in the buffer", but what does
> "ready" refer to?

Well, if you think about the write() causing a serial port to transmit
data, "ready" refers to the state of the transmitter, how ready it is to
send, in terms of how much it can accept right now without requiring
extra delay, because it's not ready for any more than that much. The
word "ready" is an adjective, describing the condition of Serial.

But really, my intention is NOT to argue for "ready" or "room" or any
other name in particular. None of them are perfect. Perhaps a long
concatenation of words could perfectly communicate the concept (even
though Arduino tends to use short and simple names). I would be
perfectly willing to change to such a verbose name too, if that's what
Cristian, Tom & the rest of the Arduino Team decide, in the interest of
future compatibility with Arduino's API.

The one and only thing I am NOT going to do is wait any longer to make
this feature available on Teensy. Long-term, I believe we all benefit
from API compatibility, even if it's difficult to decide such details.

[Randall Routh]

For my 2 cents I like "TXAvailable" or "AvailableTX".  Where "Available" reports the number of bytes available to be read from the receive buffer.  This new method reports the number of bytes available to receive bytes written to the transmit buffer.

I can see a applications/sketch that wants to continuously report data/status but does not want to stop everything to do so.  If a status report is 10 bytes and there is only 9 bytes of available transmit buffer, skip the status report until there is more space in the buffer.

--
You received this message because you are subscribed to the Google Groups "Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@arduino.cc.

--
Randall Routh
404-276-8373

[Peter Feerick]

How about writeBufferRemain (rationale: complements write function, verbose name)? Or writeBufferRoom if you want room.

Tom, I don't like transmitBufferSize as this doesn't return the size of the buffer, but how much of it is free.

Victor, I fully agree with you on verbose naming... When I initially saw room I went 'room what?'. And with regard to serial... Room for what?!

Peter

[Tom Igoe]

On Jun 30, 2014, at 5:50 PM, Peter Feerick <peter....@gmail.com> wrote:

> How about writeBufferRemain (rationale: complements write function, verbose name)? Or writeBufferRoom if you want room.
>
> Tom, I don't like transmitBufferSize as this doesn't return the size of the buffer, but how much of it is free.

Good point.

>
> Victor, I fully agree with you on verbose naming... When I initially saw room I went 'room what?'. And with regard to serial... Room for what?!

Agreed with you and Victor there, especially after some of the other comments: if it’s a not often-used function, all the more reason to name it so that its function is very apparent.

t.

>
> Peter
>
>> On 1 Jul 2014, at 3:37 am, Paul Stoffregen <pa...@pjrc.com> wrote:
>>
>> Whether we can all merely agree on a name is a good question?
>

[David Mellis]

Just to throw my two cents in, I'd go with something like "availableToWrite()" or "availableForWrite()" -- I like the parallel with available() and the use of "write" to suggest the relationship to write(). As a (relatively) rarely-used function, I agree that a verbose name is good and that room() or ready() are ambiguous. 

[Rob Tillaart]

uint32_t TxBufferFree() ?  // short and still descriptive enough ?or 

uint32_t transmitBufferFree()  // bit longer

[Randall Routh]

You don't think uint16_t will not be enough?

Randall Routh
404-276-8373

[Rich Obermeyer]

uint16_t TxBufferFree() is a great choice

[Tom Igoe]

That’s an alternative that works for me.

[Matthijs Kooijman]

> > uint16_t TxBufferFree() is a great choice

I like Free better than Available - since Available could be taken to
mean "bytes available" instead of "space available", free is less
ambiguous in this regard.

I'm wondering if Tx fits well enough in the Arduino naming convention -
WriteBufferFree refers more directly to the write() function.

Gr.

Matthijs

[Jimmy He]

+1 for uint16_t txBufferFree(). Personally I think any adjective that appears in the function makes the user feel that this is a Boolean function. My first impression on the Serial.ready() was something that tells you whether serial is ready, not how much it is.

Inviato da iPhone

[David Mellis]

Yeah, I think "write" is much better than "Tx". We try to avoid acronyms in general and the connection with the write() command is nice.

[Rob Tillaart]

+1 for writeBufferFree()

[Randall Routh]

Is this a virtual method in Stream that will be implemented in HardwareSerial?  Does this have broader application?

--
Randall Routh
404-276-8373

[Paul Stoffregen]

On 07/02/2014 11:10 AM, Randall Routh wrote:

Is this a virtual method in Stream that will be implemented in HardwareSerial?

Currently, it's not implemented at all.  We're merely talking about what name would be used, hypothetically.

I'm going to implement this on Teensy.  In fact, I already have committed it to github (using the name "room").  My hope is to publish a release candidate by Friday, ideally using a name that will be compatible with Arduino in the future.  But absent any sort of decision, I'll just make something up and hope for the best.  Real customers need to be able to avoid stalling their sketches.  I'm going to provide this functionality in my next release.

Whether Arduino ever implements this feature is a good question?

  Does this have broader application?

It's hard to imagine this anywhere, if we can't even reach consensus on the name!

Even if a name is chosen, and even if someday it's implemented in HardwareSerial and other classes, making this a virtual method in Stream would require all classes inheriting from Stream to implement it.  That seems like an incredibly distant future at the rate things are progressing... at least 2 years of requests for this feature, and so much indecision over the name.

I really have to wonder about the future of Arduino's API, as a platform, if such a minor feature is this difficult.

[Brian Cook]

> +1 for writeBufferFree()

Ditto.

[Rob Tillaart]

> making this a virtual method in Stream would require all classes inheriting from Stream to implement it. 

why not implement it to just return zero and allow it to be overruled by derived classes? Then classes do not need to implement anything...

uint16_t stream::writeBufferFree()

{

  return 0:

}

On Wed, Jul 2, 2014 at 8:55 PM, Brian Cook <bc...@rowdydogsoftware.com> wrote:

> +1 for writeBufferFree()

Ditto.

[Rich Obermeyer]

I agree this should be a big red flag.  Takes forever to get simple stuff done.  

Takes longer to get function a name than implement, test and release the code.

What complex task can be expected to complete timely.

Paul seems to be the biggest driver.

Overengineering comes to mind.

[Victor Aprea]

Rob,

Wouldn't returning UINT16_MAX be more appropriate than returning zero in the default case? That way check for "at least N free bytes in the write buffer" would always succeed.

Regards,

Vic

Victor Aprea // Wicked Device

victor...@wickeddevice.com

M: 607.745.1515

T: @vicatcu

[Rob Tillaart]

Yes Victor, that is probably a better idea: to use UINT16_MAX as a special case.

more generally:

should the return type be int or long ? and return -1  for NOT_IMPLEMENTED?

and also allowing for all kinds of other errors (negative numbers) for the Stream derived classes?

[Tom Igoe]

I’d go with long to be safe. 

[Rob Tillaart]

So the proposal for Stream level seems to be something like 

Stream.h

#define NOT_IMPLEMENTED -1L

+

Stream.cpp

long Stream::writeBufferFree()

{

  return NOT_IMPLEMENTED:

}

And in HWSerial we have the proper overruling implementation as Paul Stoffregen has written.

[Randall Routh]

I apologize for stirring this up.  I agree with the comment that more time is being spent discussing the name that to actually working on the code.  And most people, like myself, have opinions to contribute without contributing much of anything else.

I even fail to look at the existing code before running off and sharing my thoughts.  If I had spent a couple of minute to look at things at all I would have suggested the Print class and not the Stream class.  Print has most of the write methods while Stream add the read methods.  Since this is 'write' related, I should have suggested adding to the Print class, if I suggested anything at all.  Instead, I started typing before I started reading.  I am not smart enough to talk before I think, therefore, I am not smart enough to reply before I research.

I could see a time when someone is writing a driver for a display or something and wants to use Print as a parent.  The device may be slow and they may want to buffer like the serial I/O and could therefore have a similar need.  However, that is all "maybe" and "someday might".

"The Perfect is the enemy of the good"  Paul has a good solution/name/implementation.  Let's not spend endless comments on sharing with everyone what we think would make it Perfect.

Paul asked what we thought of the name.  We have all given him the benefit of our 2 cents.  If he adds another $5 he can buy a cup of Starbucks.  I will be happy with what ever he calls the method (except maybe 'RoomLeftInTheOutputBufferSoIDontNeedToWait')

Again, Paul, I apologize.

Randall Routh
404-276-8373

[Paul Stoffregen]

On 07/02/2014 05:06 PM, Dennis Lee Bieber wrote:
> If the main purpose is to avoid blocking, I'd likely look for something
> more on the order of:
>
> if (Serial.wouldBlock(size))...
>
> in which size is the length of data desired to be sent.

There are at least 3 important use cases.

#1: A fixed size message is sent if it can all go into the transmit
buffer, otherwise nothing is transmitted (until later).

#2: A very long message is transmitted repetitively (eg, DMX512
lighting) piece-wise, where each time loop() runs, as much as possible
is transmitted. An index variable is updated, to remember which portion
of the message still needs to be transmitted. Typically the rest of the
code modifies parts of that big buffer, with the expectation that the
entire thing will eventually be repeated.

#3: As much of the message as possible is sent, with the rest
intentionally discarded, and usually an error count incremented if any
data was "lost".

[Cristian Maglie]

In data mercoledì 2 luglio 2014 20:51:55, Paul Stoffregen ha scritto:
> I'm going to implement this on Teensy. In fact, I already have
> committed it to github (using the name "room"). My hope is to publish a
> release candidate by Friday, ideally using a name that will be
> compatible with Arduino in the future. But absent any sort of decision,
> I'll just make something up and hope for the best. Real customers need
> to be able to avoid stalling their sketches. I'm going to provide this
> functionality in my next release.

It looks like writeBufferFree() has most consensus, so let's go with that.
I'll guarantee that this will be the name used in Arduino API (when/if it will
be implemented).

C

[Paul Stoffregen]

On 07/03/2014 01:15 AM, Cristian Maglie wrote:
> It looks like writeBufferFree() has most consensus, so let's go with
> that. I'll guarantee that this will be the name used in Arduino API
> (when/if it will be implemented). C

Thanks! I've updated my code to writeBufferFree().

If you decide to include this in Arduino someday, just say the word and
I'll submit a pull request. The implementation is really very simple.

[Tyler]

The issue with using a boolean return value for something like
.wouldBlock(size) is that you wouldn't be able to figure out how much
data to send and how much to save to send later. You'd have to keep
submitting smaller and smaller "size" parameters until the function
returned true.

[Bill Perry]

I believe that this issue needs to be solved at a higher level in a more generic way
rather than just quickly hacking in a new function to solve this one specific need.

To me this specific issue is an issue of blocking vs non blocking i/o.
I also think that the the TX and the RX side should be
symmetrical and use a similar naming conventions which today is not the case.

Today we have a situation where reads are non blocking and writes are blocking.

What is missing in the library is the ability to control or configure the library's behavior
from the sketch.

What I'd much rather see would be some sort of i/o control call (or maybe a parameter on begin())
that would allow the sketch to configure whether the i/o was blocking or non blocking on each direction.
So for example, the default could be to have the way it works now for backward compatibility.
If the TX/write i/o was configured for non blocking, then the write() function would return immediately
if the buffer was full. The return code could be 0 or -1 (to match the RX non blocking status)
to indicate that the write() didn't not handle the character because the i/o would have blocked.

Also, there are times where blocking reads are perfectly acceptable and actually desired.
In that case the sketch could set the RX/read mode to blocking and a call to read() would block
until a character was available.

There may also be cases where the message is "atomic" and you may want the write()
to return immediately if it can't buffer the full message.

The point is the sketch needs the ability to control the blocking/non-blocking behavior
of the TX and RX directions. Once you can do that, things get simpler and it
starts to work more like normal i/o calls in real operating systems.


While not my preferred approach,
As far as naming of function for querying the TX output status goes for a hacked in function
that returns in whether the write() would block,
why not converge onto the same naming that was used on the RX side?

Today the RX side uses available().
My suggestion would be to deprecate available()  (leave it there for backward compatibility, but no longer document it)
and then have RXavailable() and TXavailable()
where RXavailable() returns the RX characters available, and TXavailable() returns the amount of buffer space
available.
That way the functions are clean/simple, and there is a consistency between the RX and the TX side.

All these somewhat "random" function names, like room(),  writeBufferFree() wouldBlock() and others just leave my head shaking
since they all seem kind of vague and there is no consistency with the RX side.

I think consistency is important and that some of the other functions mentioned like wouldBock() are
applicable to both the TX and RX side.

As far as the final name goes: writeBufferFree() ???
To me that sounds like a function that is going to free a buffer.

Or instead of a specific function to get the tx buffer information
consider adding a real i/o control interface as a single or pair of API functions
that could be used to query/set library parameters similar to the way i/o control
calls work in normal OS drivers.
So the sketch could query the library for the free room in the buffer
by simply making query call and asking for the desired parameter.
The sketch could also configure the library behavior using this interface.
This adds at most 2 API functions, but then can be infinitely expanded
to provide all kinds of information.
This keeps the API from exploding with all the "random" function names.
i.e. by adding just 2 functions, the sketch could then set or get library
specific parameters, behaviors, status, etc...

--- bill

On Monday, June 30, 2014 3:58:18 AM UTC-5, paul wrote:

We've talked about this API many times, always with a consensus it makes sense.  Recently, I've had more users request a way to avoid blocking in Serial.write().  I'm implementing it now on Teensy, and I hope someday it'll be implemented in Arduino's core library.

My hope is we can agree on the name for this function, so when/if it's ever implemented in Arduino's core library, we'll end up with the same name.  Long-term, I'd like to stay as compatible with the Arduino API as possible.  Yesterday I committed code with "room", but since it's only 1 day old, changing is still pretty easy.

Previously, I had preferred Serial.ready().  This forum thread swayed me to Serial.room().

http://forum.arduino.cc/index.php/topic,165574.0.html

Really, I don't care what the name end up being.  Feedback I've heard from several people is a design for a longer, more descriptive name.  Suggestions have included Serial.availableWrite(), Serial.writeAvailable(), Serial.readyWrite(), Serial.TxAvailable(), and other variations.  Any of these names is fine.  Agreeing on a name, so all boards implementing this remain compatible, seems like a good idea.


Regarding the feature itself, the last time I brought it up was item #3 on this API wish list.

https://groups.google.com/a/arduino.cc/d/msg/developers/PcOW-MCuDHc/KxLcOKLlNhoJ

There are at least 2 pending requests on the issue tracker:

https://github.com/arduino/Arduino/issues/672

https://github.com/arduino/Arduino/issues/1408

[Paul Stoffregen]

Today we have a situation where reads are non blocking and writes are blocking.

What is missing in the library is the ability to control or configure the library's behavior
from the sketch.

This sort of symmetry and configurability, while beautiful from a pure software design perspective, I believe would be a terrible mistake for Arduino's mission to be accessible for novices.

Arduino's approach, where reading is non-blocking and writing is blocking, has proven to be very easy for ordinary people to understand.  Yes, it lacks symmetry, but it has the much more important quality of simplicity for people without professional programming experience.  It's a perfect example of "if it ain't broke, don't fix it".

Or instead of a specific function to get the tx buffer information
consider adding a real i/o control interface as a single or pair of API functions
that could be used to query/set library parameters similar to the way i/o control
calls work in normal OS drivers.
So the sketch could query the library for the free room in the buffer
by simply making query call and asking for the desired parameter.
The sketch could also configure the library behavior using this interface.
This adds at most 2 API functions, but then can be infinitely expanded
to provide all kinds of information.
This keeps the API from exploding with all the "random" function names.
i.e. by adding just 2 functions, the sketch could then set or get library
specific parameters, behaviors, status, etc...

I'm sure you've seen Arduino's API style guide, but here's the link, just in case.

http://arduino.cc/en/Reference/APIStyleGuide

POSIX's fnctl() & ioctl() and WIN32's DeviceIoControl() are APIs that Arduino should not mimic.  Just imagine asking an Arduino user to read any of these pages!

http://pubs.opengroup.org/onlinepubs/009695399/functions/fcntl.html
http://msdn.microsoft.com/en-us/library/windows/desktop/aa363216%28v=vs.85%29.aspx
http://www.xml.com/ldd/chapter/book/ch05.html
http://man7.org/linux/man-pages/man2/ioctl_list.2.html

[Tom Igoe]

Paul posts for me on this topic.  At this point there is a description of the function, and there’s an agreed-upon name. I think we’ve fulfilled what Paul was looking for when he started the topic (correct me if I’m wrong, Paul), and we can move onto other matters.

t.

[Cristian Maglie]

In data giovedì 3 luglio 2014 21:58:25, Paul Stoffregen ha scritto:
> If you decide to include this in Arduino someday, just say the word and
> I'll submit a pull request. The implementation is really very simple.

Hi Paul,

I see that you already commented on the old issues so, yes, go ahead an post a
pull request for that.

Thank you! :-)

C

[Bill Perry]

On 07/05/2014 09:06 AM, Tom Igoe wrote:

Paul posts for me on this topic.  At this point there is a description of the function, and there’s an agreed-upon name. I think we’ve fulfilled what Paul was looking for when he started the topic (correct me if I’m wrong, Paul), and we can move onto other matters.

t.

On Jul 5, 2014, at 5:28 AM, Paul Stoffregen <pa...@pjrc.com> wrote:

Today we have a situation where reads are non blocking and writes are blocking.

What is missing in the library is the ability to control or configure the library's behavior
from the sketch.

This sort of symmetry and configurability, while beautiful from a pure software design perspective, I believe would be a terrible mistake for Arduino's mission to be accessible for novices.

I realize that I'm coming into the discussion very very late (hopefully not too late),
but I do think it is worth taking a second look at these function names and how they work.

Ignoring the reconfigurability of the library for a moment (as that is a separate/different set of features)

Trying to keep not only naming of the API functions consistent but their return values is bad? Really?
So using available() for the read() side and  writeBufferFree() for the write() side is better
than using names that are more consistent with each other and more clearly indicate their intended use is worse?
All I can say is wow.

I totally agree with you that novices need all the help they can get.
I think creating a consistent API, which includes the naming of the functions and their behavior,
goes a long way to helping people understand the API.
So my view would be to leave available() for backward compatibility and then
create two new functions that are consistent with each other in terms of usage, naming, and return values
that can be used to extract how many bytes are are available on the read() side and how many bytes are remaining
in the write() output buffer.

While anything can be made to work (some of the Windows APIs show that),
and a function name is just a name, lots of API functions of similar sounding
or somewhat disjoint names can turn an API into a giant bowl of spaghetti.

I had suggested RXavailable() and TXavailable() but I think any two names for these functions
that are consistent would be better than the current names.
Perhaps even ReadAvailable() and WriteAvailable() ??


One a related note,
also keep in mind that the library API now also should have a function to be able to query how large the buffers are
because they are not a consistent size across the all the processors nor the same size in either direction.
This may be necessary to give more advanced code the ability to break its messages into different sizes to ensure no blocking
with messages on the write() side or buffer overflows on the read() side.

Arduino's approach, where reading is non-blocking and writing is blocking, has proven to be very easy for ordinary people to understand.  Yes, it lacks symmetry, but it has the much more important quality of simplicity for people without professional programming experience.  It's a perfect example of "if it ain't broke, don't fix it".

But I believe that while what is there today "works",  it is broken or at least missing significant functionality.

I'm not buying the "simplicity" argument. We see many many posts in the forums
where users are confused by the data they are getting back from read() because
it is not blocking.
Yes they finally grasp that it is non blocking but the issue comes up quite frequently.
And then we also see posts on the other side where they are getting unexpected pauses
on output since the output is semi-blocking or they output something and expected
it to be fully transmitted when the write() returned.

Many novices/newbies are not ready to think about threads and the coding required
to deal with non blocking vs blocking i/o.
So I think many novices could benefit from being able to set the read() to be blocking.
It simplifies their code to just calling read().
It allows them to get up and do something without having to write a wrapper to do the blocking.

In my view, Arduino is about simplicity. Allowing users to jump right in and not have to dig
in too deep before getting something up and running is what Arduino is all about.

Some of these Arduino novices are coming from other environments like python, or java
which do have the ability to create blocking and non blocking io streams.

So I believe that being able to specify the blocking/non-blocking behavior
would be quite useful even it was just from the begin() call.
At least if it was something they set or could set, they could then see and immediately start to understand the different
modes of operation and not be surprised by they way it works, like they are today.
Having defaults is good, but having the ability to change or tune away from those defaults is better since
not all applications can use the defaults.

The entire point of creating this new tx buffer size remaining function,
is primarily due to the fact that the more advanced user is unable to alter the behavior of the library write() function from
semi-blocking to completely non blocking.

Bottom line is that sometimes it is desirable to have the read()/write() functions block and sometimes
it is desirable to have them always return immediately.
In my view, that is the real issue here. Not that there is a missing function to return the remaining space in a buffer.

So in terms of the overall semantics of the API
(an API where you can't control or alter the behavior of the library)
this API definitely doesn't look or work anything like what has been done for more than 40 years in modern operating systems
nor what is being done for i/o in more modern languages like python or Java.
I would argue that the experience of those operating systems, languages and APIs got it right over the decades and that Arduino is getting it wrong....

Why create a set of semantics that are nothing like what has been in modern operating systems for decades?
Why step out and be different just for the sake of being different?

I believe that consistency is a very good thing.
I also think that that consistency is not the enemy of simplicity and ease of use.
It doesn't have to be complicated, nor does it have to be instead of having "available" functions.

So my overall view is that both having the "available" functions and the configuration capability are probably needed
(Although if you got the configuration capabilities and functionality right you wouldn't need the "available" functions)
That said, I do think that their should be more thought and attention paid to ensuring a consistency
between the names and return values of  API functions like the "avaialble" functions for the read() side and write() side of the API.



--- bill

[Bill Perry]

This was touched on earlier in the discussion but I'm bringing up again:
Why should this be functionality only for HardwareSerial?

It would make more sense to put this function up in stream where available() is.
As also pointed out earlier, it has potential ripple effects to other existing libraries
and APIs depending on how it is implemented.
There are also potential implications to classes like Print

What this is pointing out is that simple "hacks" are not always so simple.
I understand there is a desire to try to move fast but I'm not happy
to see a hack like this get tossed in this way: where it is just for one specific library.

I believe that this issue and need for addtional functionality
exists for more than just HardwareSerial and therefore
should be solved in a larger way in some class outside of HardwareSerial so that
other libraries can also offer this functionality and to ensure that all libraries handle
it the same way.

In other words, in my mind if there is a genuine need for this feature/capability, it should be important
enough to add it in a way that works for other streaming i/o devices/libraries rather
than just hacking it into a single library.

--- bill

[Tyler]

The issue I see with setting blocking/non-blocking behavior in the begin() function, is that sometimes you want to block, and sometimes you don't, within the same sketch. I have a very large Arduino project with multiple functions for the Serial ports - when a certain pin is high, the Serial RX is for configuration messages, and the configurator must wait for a complete message before executing the command. However, when that pin is low, the Serial RX becomes nothing more than a forwarder and doesn't care about receiving complete messages before it forwards them. So in this case, sometimes I'm using readBytes() in a blocking style, waiting for commands, where in others I'm using read() only when there are bytes available.

So my suggestion would be a separate function for non-blocking behavior. Something like readAsync(), which can do the things you mentioned below. This way it doesn't break existing code or make the basic functions confusing for beginners.

Hell, we might as well add a callback function that gets called with the data when there is some... that could be useful instead of everyone trying to implement their own polling solutions. Would asynchronous callbacks even make sense on a platform like Arduino?

-Tyler

[Paul Stoffregen]

On 07/05/2014 01:15 PM, Tyler wrote:

Hell, we might as well add a callback function that gets called with the data when there is some... that could be useful instead of everyone trying to implement their own polling solutions. Would asynchronous callbacks even make sense on a platform like Arduino?

Well, I wasn't going to bring this up, at least not for quite some time.

Today we have serialEvent() in the official Arduino API.  The serialEvent callbacks are only made each time loop() completes, which minimizes compatibility issues, but doesn't always give responsive results.

The other extreme involves making callbacks from interrupt context.  There are 2 huge problems with interrupt context.  First, properly managing shared data is a topic even experienced programmers find difficult.  Second, many of Arduino's functions and libraries aren't reentrant or otherwise fail if used from both main program and interrupt context.

I recently implemented serialEvent on Teensy, and in doing so, I'm starting an experiment to try to achieve something roughly in the middle of these 2 extremes.  The idea isn't new, and I believe it was discussed on this mail list a couple years ago, of course with much controversy, like almost everything discussed here.  The basic idea is to do the callbacks from yield().  Teensyduino calls yield in all blocking operations, like delay() and Serial.write().  I implemented a static flag inside yield() to prevent recursive calling, for example, if anything you do in serialEvent calls something which calls yield again.

I'm probably going to expand this experiment with a timerEvent API, offering 1 ms resolution.  The millis interrupt will merely set flags.  The callbacks will never be made from interrupt context.  All callbacks will be made from yield(), which is a trade-off that avoids the worst issues with concurrency & data sharing, because the compiler assumes called functions can read or write any memory locations.  It's also a trade-off for latency, where CPU-hogging code can still delay the callback, but at least callbacks will be timely for the common case where sketches do lots of I/O.  I might add more yield() calls in other cases, like when Serial.available() returns zero, to improve responsiveness.

Why am I bothering to write this?  Long term, Arduino needs to facilitate more concurrency.  I've spent a lot of time looking at various approaches, which I'm not going to enumerate in this already-too-long message.  My belief is event callbacks, called from main program context, are by far the most novice-friendly way to provide concurrency.

This isn't a proposal for Arduino.  Right now, it's merely an experiment I'm trying on Teensy to implement a lower latency serialEvent callback.  When/if I expand to timerEvent, I'll probably ask here if there's any interest to tentatively agree on the API, in the interest of future compatibility if Arduino ever decides to implement it.  I'm really not looking forward to making any such proposal.  Just imagine how hard that will be, compared to this fairly simple write status!

But for now, I'm going to put together a simple pull request for writeBufferFree, and over the next few weeks I'm going to focus on SPI transactions, because we all desperately need the many SPI-based libraries to be able to work together.

[Rich Obermeyer]

Paul, your doing the right thing.

Keep up the great work.

[Paul Stoffregen]

> Why should this be functionality only for HardwareSerial?

It probably also makes sense in other places, but it's most frequently
requested for HardwareSerial, so that seems like a good place to
implement it first.

> In other words, in my mind if there is a genuine need for this feature/capability, it should be important
> enough to add it in a way that works for other streaming i/o devices/libraries rather
> than just hacking it into a single library.

On Teensy, I've already added it in HardwareSerial, USB Serial, and
emulated USB Serial. I'm about to prepare a pull request for Arduino
1.5.x to add it in HardwareSerial. I may implement it for other
objects, but after this pull request for 1.5.x, I'm planning to move on
to (or get back to) SPI transactions and some other Teensy-specific
features that I feel are far more important.

Bill, in principle, I get what you're saying and I believe I probably
even agree. In reality, somebody actually has to write and test the
code in all those other Print/Stream based libraries. Are you going to
take that on?

[Bill Perry]

For this single write() "available" function "hack" I probably would take it on.

Although my suggestion would be to add a new virtual function in Stream
which can be overridden by the lower level classes like HardwareSerial.

I would suggest that the default write() "available" function in Stream
return that there was always a large amount of space available which
is not really correct or optimal but would allow some level of compatibility with newer code that wants
to use the function by allowing characters to be pushed out while potentially blocking when buffers fill.
The alternative to that would be to not have a default function to let the link fail, or have
the default function return that there is no room, both of which I think are worse.
Forcing the new function by allowing the linker to fail, would be the "break the world" approach,
which would get all the 1.5x code that used Stream updated, but I'm not really a fan of
"break the world" solutions. But if that were the decision, it would better to do something like that now
vs the way 1.0 was handled where it was done with the final release. Plus this wouldn't break everything
like 1.0 did, it would only break the 1.5x 3rd party code that used Stream.

The existing code that you have already done for teensy,
would not change at all other than their function declarations - or the names if the "available" names were changed from their existing names.

In other words, what I'm looking for is creating interfaces that are not proprietary to libraries and
can be used the same way with the same name across multiple libraries and the best way to do that is
to define the interface in a class that is outside the actual library.

My fear is if this is not added to Stream, we run the risk of ending up with a situation like what we have with the hd44780 LCD libraries.
Those are quite the mess. Different library writers extended the library in different ways so now when you move
across different libraries or libraries with different interfaces like i2c or shift registers,
you can end up with different capabilities or functions that have the same functionality but have different names or use different parameters.

Putting the write() "available" function in Stream were the read() "available" function already is seems to make sense to me
and would ensure that once implemented, all the sub classes would implement it the same way and have the same name.


In terms of handling non blocking i/o in Print, that is a much bigger issue no matter where/how this "available" function is implemented.
The biggest issues for something like Print is that currently it is a blocking interface and its interface to the lower level
class is a 1 character at time interface.
The current Print class API simply does not have a way to implement non blocking without quite a few modifications.

--- bill

[Rob Tillaart]

> I would suggest that the default write() "available" function in Stream
> return that there was always a large amount of space available which
> is not really correct or optimal but would allow some level of compatibility with newer code that wants
> to use the function by allowing characters to be pushed out while potentially blocking when buffers fill.

imho the "write() available" should by default return some constant UNKNOWN (e.g. -1) if it is not known. 

A derived class can override this default behaviour if it knows how to give a meaningful value. 

[David Mellis]

I think something like this makes a lot of sense. Cooperative multitasking (is that the right name for this?) feels like it should provide enough responsiveness to be useful in a lot of situations without the additional complexity of interrupt-based events.

[Matthijs Kooijman]

Hey Rob & Bill,

First off - Bill suggests putting the writeBufferFree() function in
Stream along with available(), but putting it in Print (which has the
write() function) makes a lot more sense to me (and I think I've seen
other mention this as well).

I guess we should opt for combining these options - return a specific
constant to indicate that writBufferFree is not implemented, but choose
the value as a high value (e.g 2^16-1). This means that callers can
provide backward compatibility with old Print implementations by
checking for this constant. Alternatively, they can just ignore this and
see a lot of free space in the buffer, which is often also fine (with
the risk of having things block when not expected, but that's the best
we can do in this situation).

Gr.

Matthijs

[Paul Stoffregen]

>> imho the "write() available" should by default return some constant UNKNOWN
>> (e.g. -1) if it is not known.
>> A derived class can override this default behaviour if it knows how to give
>> a meaningful value.
> I guess we should opt for combining these options - return a specific
> constant to indicate that writBufferFree is not implemented, but choose
> the value as a high value (e.g 2^16-1).

Another possibility might be a default of 1, causing the most
latency/blocking sensitive applications to write 1 byte at a time,
assuming they check writeBufferFree() to limit themselves to only data
which can transmit with minimal delay. A large default value might be
good for data transmit efficiency, but it would likely cause such
programs to unnecessarily block, which seems like a bad outcome where
the author when to extra effort to use writeBufferFree(). When someone
uses this function, it's very likely they care more about their other
code continuing to respond to sensors and update real-time devices like
motors, where rapid hardware response is far more important than
efficiently transmitting data.

I personally do not have a strong opinion whether writeBufferFree()
should be in the Print class, and if it is, what the default should be.
But I do believe a positive integer default would be a much lower burden
to authors of sketches and libraries, who are unlikely to check for a
special value, especially if they test against an object which
implements writeBufferFree(). When they share their code, others who
modify it to use a different Print class object would likely be unable
to resolve such problems. A special "not implemented" value, which
causes things to fail, seems like a poor choice for a platform where
where source code is shared among novice users.

If writeBufferFree() isn't implemented in the Print class now, perhaps
it might be worth reconsidering later, when/if demand from users and
library authors begins to grow?

[Bill Perry]

On 07/07/2014 05:13 AM, Paul Stoffregen wrote:
>>> imho the "write() available" should by default return some constant UNKNOWN
>>> (e.g. -1) if it is not known.
>>> A derived class can override this default behaviour if it knows how to give
>>> a meaningful value.
>> I guess we should opt for combining these options - return a specific
>> constant to indicate that writBufferFree is not implemented, but choose
>> the value as a high value (e.g 2^16-1).
>

> Another possibility might be a default of 1, causing the most latency/blocking sensitive applications to write 1 byte at a time, assuming they check writeBufferFree() to limit themselves to only data which can transmit with minimal delay. A large
default value might be good for data transmit efficiency, but it would likely cause such programs to unnecessarily block, which seems like a bad outcome where the author when to extra effort to use writeBufferFree(). When someone uses this function,
it's very likely they care more about their other code continuing to respond to sensors and update real-time devices like motors, where rapid hardware response is far more important than efficiently transmitting data.

A default of one doesn't really work either.
Ok it might kind of sort of work in some cases but not always and I think those
types of solutions are not real solutions and create subtle issues.

Using 1 is essentially no different than using a very large number.
The difference is are you telling the caller he has a very large buffer available vs you are
telling the caller he ALWAYS has room for 1 more.
Both will result in potential output blocking.
The biggest difference is that it results in blocking for a shorter amount of time
when the data is sent down since it would only be blocked for a single byte time.

Is there a way to detect of the writeBufferFree() function has been declared in the class?
(I'm not a C++ expert)
Can you use some sort of virtual function and then check to see if it has been overridden by the
library to allow the sketch to detect if the library supports the function?

So maybe the ideal would be to put it into Stream and the default function returns 1
(or an error status) if the sub class doesn't declare the function,
but then also have the ability for the sketch to be able to detect if the sub class
didn't declare the function to know that the functionality doesn't exist?

But then I'm not sure what the sketch would do differently from the 1 at time
write() if the functionality didn't exist........
But at least the sketch could know.

In the case of the default function in Stream returning an error status, if
the sub class doesn't re-define the function, the sketch
would know that that the "available" function is not supported and
could act accordingly.

>
> I personally do not have a strong opinion whether writeBufferFree() should be in the Print class, and if it is, what the default should be. But I do believe a positive integer default would be a much lower burden to authors of sketches and libraries,
who are unlikely to check for a special value, especially if they test against an object which implements writeBufferFree(). When they share their code, others who modify it to use a different Print class object would likely be unable to resolve such
problems. A special "not implemented" value, which causes things to fail, seems like a poor choice for a platform where where source code is shared among novice users.
>

What I used in openGLCD is negative return values always means an error status.
Then there are multiple values defined for various errors.

There is already precedence for this in the read() routine.
read() returns a negative value (currently -1) if there is no character available
vs the actual character.

Given that today the writeBufferFree() routine does not exist and is not used
by any code, there doesn't seem to be any sort of burden being placed on users.
It becomes a documentation issue.
Also writeBufferFree() is for doing a more sophisticated i/o and I think
that users using it are going to be more advanced users and will be less
likely to have issues or confusion with the dual type of status/data being returned.

Alternatively, if you want to be more C++ like and enforce better type checking,
then change the function to always return an enum status and have the user
pass in a pointer of where to store the desired data.

On 07/07/2014 02:48 AM, Matthijs Kooijman wrote:
> Hey Rob & Bill,
>
> First off - Bill suggests putting the writeBufferFree() function in
> Stream along with available(), but putting it in Print (which has the
> write() function) makes a lot more sense to me (and I think I've seen
> other mention this as well).

Print is a tricky one due to its history and backward compatibility issues with
existing libraries.
It existed before Stream which was introduced in 0019
Prior to stream there really wasn't any "i/o" type of class.

While I do think that having the "available" functions where write() is
does make some sense, I'm not sure moving it/them into Print is a good thing.
It would seem to make more sense if a write() were added into Stream.

Currently, Stream inherits from Print and Print is for output only whereas Stream is bidirectional.

There are many output only libraries that use print that have no need/use for input or the Stream
capabilities.
For example, graphic libraries, LED libraries, LCD libraries, etc...

And then there is no requirement for Print functionality if doing Stream i/o.


If things are flipped and Print inherits from Stream
(which it might do if it plans on using the Stream "available" functions) then many
libraries would break, due to the pure virtual functions in Stream as they don't
have any of them since they were not needed, unless Print declared some stubs
for them - which it probably would do.

I think what all this is pointing to is that this is all part of how i/o is handled in Arduino.
Do to the way i/o has evolved on Arduino, vs being actually planned/designed,
there is no simple quick fix and and there will need to be some deep thought on how to
make it "grow up" and not break all the existing code.

There needs to be an i/o class like Stream, perhaps it is Stream, that implements the i/o
and something like Print should just be a user of i/o and not be implementing any of the i/o
functionality.


--- bill

[Matthijs Kooijman]

Hey Bill,

> Using 1 is essentially no different than using a very large number.
> The difference is are you telling the caller he has a very large buffer available vs you are
> telling the caller he ALWAYS has room for 1 more.
> Both will result in potential output blocking.
> The biggest difference is that it results in blocking for a shorter amount of time
> when the data is sent down since it would only be blocked for a single byte time.

Agreed. And as an extra problem, a default value of 1 cannot be used to
detect if the return value is meaningful or not (a value of MAX_INT can
be expected to never occur in practice).

> Is there a way to detect of the writeBufferFree() function has been declared in the class?
> (I'm not a C++ expert)
> Can you use some sort of virtual function and then check to see if it has been overridden by the
> library to allow the sketch to detect if the library supports the function?

Nope. You can of course add a "virtual bool supportsWriteBufferFree()"
function, but that is ugly and cumbersome...

Also note that for code that uses Stream/Print references, you can't
resolve this at compiletime - it might get both a HardwareSerial object
(which iplements writeBufferFree()) and a
PrintSubclassWithoutWriteBufferFree object at other times.

> In the case of the default function in Stream returning an error status, if
> the sub class doesn't re-define the function, the sketch
> would know that that the "available" function is not supported and
> could act accordingly.

Exactly.

Gr.

Matthijs

[Bill Perry]

So what about using Jan's solution??:
https://github.com/arduino/Arduino/issues/1930

It provides full backward compatibility, moves the API to Stream so it is implemented
the same for all libraries, gives the sketch the ability to tell if it is implemented,
AND... uses consistent meaningful names.


--- bill

[Cristian Maglie]

Hey Bill,

I've just read the whole discussion (in mailing list and github) you raised a
bunch of interesting points.

1) About the name admittedly your arguments convinced me, now I think we
should change the name and use the prefix "available..." to keep consistency.
Jantje called it "availableInSendQueue", but IMHO we should use "write"
instead of "send" to keep a relationship with the write() method too.

Said that I'll go for the more simple "availableForWrite" so the API so far
will look like:
...
Serial.available()
Serial.availableForWrite()
...
What do you think?

Well... before going on there is a problem... I've already chosen
"writeBuffersFree" and promised to not change it, changing it now may sound a
bit weird (at the least). Is there anyone concerned if I decide to change the
name but for a good reason? Paul, in particular, you asked for a quick
decision on that, changing it now (a few days after) may cause you much
troubles?

2) I like the idea to add the new method to the Print class and provide an
"interface" for the classes that extends Print (and Stream).

This can be easily done after Paul's pull request (or even directly by Paul in
his pull request).

3) I'm not convinced about renaming the method "available()" to
"availableForRead()" for the sake of API simmetry.
Arduino's users are deeply used to "available()" IMHO we cannot change it.
Even deprecating "available()" in favor of "availableForRead()" doesn't work
for me, deprecating means that the method should not be used and this implies
that a lot of sketches, examples, documentation, tutorial, etc. should be
updated. Honestly I don't see any reason to do so.

C

[Paul Stoffregen]

> Well... before going on there is a problem... I've already chosen
> "writeBuffersFree" and promised to not change it, changing it now may sound a
> bit weird (at the least). Is there anyone concerned if I decide to change the
> name but for a good reason? Paul, in particular, you asked for a quick
> decision on that, changing it now (a few days after) may cause you much
> troubles?

Changing now is only a minor inconvenience. I'm happy to change. ;)

My hope is the name can be stable by the 1.5.8 and/or 1.0.6 release
(even if it's not back-ported to 1.0.6)

None of these lengthy names are my personal preference, but I'm not in
the mood to argue the name.....

> 2) I like the idea to add the new method to the Print class and provide an
> "interface" for the classes that extends Print (and Stream).
>
> This can be easily done after Paul's pull request (or even directly by Paul in
> his pull request).

The default return value is still an unresolved question.

I believe it should be 1. I know Bill, Jan, Matthijs and others will
probably disagree. Let me quickly explain my thinking...

Arduino is a platform where novice users combine libraries and examples,
usually with small modifications and bits of their own code, to build
projects far more sophisticated than they could have accomplished "from
scratch". Imagine a beginner who uses a brushless motor control library
that uses a Stream reference/pointer to receive commands and report
motor position. The library uses this new function, so it can monitor
the 3 hall effect sensors without terrible latency waiting for
Print::write(). It has a buffer built inside the library. The novice
user connecting the brushless motor has no knowledge of any inner
workings inside this hypothetical motor control library.

But this user wants to control the motor wirelessly, so they edit the
example from "Serial" to "myRF22", which is a communication library not
yet implementing the new function. If the default value is some large
number, the motor control library will suffer too much latency to
control the motor while the communication library blocks. If the
default is some error number, odds are good the library author only ever
tested on Arduino Uno and never added code to check for that error. But
if they do, what are their options? The motor still needs to be
controlled and the status messages still need to be sent over whatever
communication link the user needs. They're still going to need to fall
back on some default in their code. Sending only 1 byte is the safest
default, when you can't know. It's the most likely to allow projects to
actually work, because if write() does block, the latency will be the
smallest possible.

Of course, other people feel a large value, or an indication of an error
should be a default. I'm sure there will be many arguments for an error
value, but think about cases like this hypothetical pairing of a motor
control library and a wireless communication library. It's easy to talk
about APIs in terms of computer science, especially in the context of
direct use by experienced programmers, but I'm far more concerned with
indirect use (buried inside complex libraries) by novice end users. My
hope is for a default that helps real projects work as well as
reasonably possible.

I'm afraid this default value matter is also going to have to come down
to a final decision that can't please everyone. For a pull request to
have this Print class addition, a decision on the default value is needed.

[Rich Obermeyer]

I am disappointed that this is "still" being debated. I saw votes for a lot of names and tons of functionality options which pretty much all conflicted.
I saw no votes for the choices you are trying to settle on. your putting the burden on Paul who has been switching it all over the place just to be nice and no doubt has finished his code long ago and has moved on to getting other pressing issues done.

Not to confuse the issue but from my perspective expecting the core to do all this for all real time requirements seems a huge stretch anyway. There is seldom a perfect solution.

[Bill Perry]

On 07/14/2014 09:02 AM, Paul Stoffregen wrote:
>
>> 2) I like the idea to add the new method to the Print class and provide an
>> "interface" for the classes that extends Print (and Stream).
>>
>> This can be easily done after Paul's pull request (or even directly by Paul in
>> his pull request).
>
> The default return value is still an unresolved question.
>
> I believe it should be 1. I know Bill, Jan, Matthijs and others will probably disagree. Let me quickly explain my thinking...
>
> Arduino is a platform where novice users combine libraries and examples, usually with small modifications and bits of their own code, to build projects far more sophisticated than they could have accomplished "from scratch". Imagine a beginner who
> uses a brushless motor control library that uses a Stream reference/pointer to receive commands and report motor position. The library uses this new function, so it can monitor the 3 hall effect sensors without terrible latency waiting for
> Print::write(). It has a buffer built inside the library. The novice user connecting the brushless motor has no knowledge of any inner workings inside this hypothetical motor control library.
>
> But this user wants to control the motor wirelessly, so they edit the example from "Serial" to "myRF22", which is a communication library not yet implementing the new function. If the default value is some large number, the motor control library will
> suffer too much latency to control the motor while the communication library blocks. If the default is some error number, odds are good the library author only ever tested on Arduino Uno and never added code to check for that error. But if they do,
> what are their options? The motor still needs to be controlled and the status messages still need to be sent over whatever communication link the user needs. They're still going to need to fall back on some default in their code. Sending only 1 byte
> is the safest default, when you can't know. It's the most likely to allow projects to actually work, because if write() does block, the latency will be the smallest possible.
>
> Of course, other people feel a large value, or an indication of an error should be a default. I'm sure there will be many arguments for an error value, but think about cases like this hypothetical pairing of a motor control library and a wireless
> communication library. It's easy to talk about APIs in terms of computer science, especially in the context of direct use by experienced programmers, but I'm far more concerned with indirect use (buried inside complex libraries) by novice end users.
> My hope is for a default that helps real projects work as well as reasonably possible.
>
> I'm afraid this default value matter is also going to have to come down to a final decision that can't please everyone. For a pull request to have this Print class addition, a decision on the default value is needed.

I totally agree with this last statement about "can't please everyone".

The real issue how to create a default function for functionality that doesn't exist.
That is the crux of the problem.

If the main code truly needs non blocking i/o and is written to limit its i/o
to ensure blocking doesn't ever happen, then, depending on the message sizes, the timing of sending
those messages, and the underlying buffer down in the i/o library and the speed of the i/o channel,
returning a value of 1 when the "available" function can still cause the code fail or even
worse just act "wonky" and work sometimes but have "odd" issues.

While I agree that it would be nice to have a default function that would try to allow ported code to run
when the underlying functionality doesn't exist; however, doing so comes with a cost.

So if we look at your example of moving some code from running on a library that supports the "available"
function to one that doesn't and the default function returns 1 then:

"simpler" code that fragments data using the "available" function might
kind of sort of work but might not, or worse it might work with intermittent issues
that are difficult to track down.

"smarter" code would have no way of detecting that the needed functionality is missing
and more gracefully failing with some sort of indication if it can't work without the new functionality.
It would be pushed down into that "kind of sort of working" state as well.

If the return value from the default function was some kind of status indicating
that the functionality doesn't exist,
then the new code being written to use this (we are talking about affecting code not written yet)
could be written to be a little smarter to default to using 1 or posting some kind of error indication
when the functionality doesn't exist. This gives the option to the author of how to handle the
condition vs silently tricking the code using the API into believing that using a size of 1 will result
in no blocking.

Since we are talking about affecting code that isn't written yet, I believe that this is more of a
documentation issue. In my view, proper documentation
would include some sort of "best practices" guidelines or example code.
I'm sure if this was cleanly documented with an example, that users would quickly grasp
how to use an available function that returns an error status when the functionality doesn't exist.
And in many cases users would probably just lift the example code which could be written with comments
clearing indicating where they should insert their code to either revert to using 1 with a risk
of blocking or to call their "panic" function in their code to fail a bit more gracefully.

I like giving the code author the ability to choose how to handle missing functionality
vs taking that option away and trying to come an implementation that make a guess
as to what might work when the functionality is missing.

I'm all for trying to make things "just work" if possible, but in this case, there does
not appear to be an obvious way to make it "just work" so the question becomes:

Is it desirable to create an API that is knowing defined in a way that sometimes lies to the caller
and does not have a way to detect that the underlying functionality of the API function is missing/not available?

--- bill

[Cristian Maglie]

On 07/14/2014 09:02 AM, Paul Stoffregen wrote:

> The default return value is still an unresolved question.

>

> I believe it should be 1. I know Bill, Jan, Matthijs and others will

> probably disagree. Let me quickly explain my thinking...

Oh, sorry I thought you agreed on the return value too. If you need to send the pull request go ahead without the Print addition while we try to figure it out.

> But this user wants to control the motor wirelessly, so they edit the

> example from "Serial" to "myRF22", which is a communication library not

> yet implementing the new function. If the default value is some large

> number, the motor control library will suffer too much latency to

> control the motor while the communication library blocks. If the

> default is some error number, odds are good the library author only ever

> tested on Arduino Uno and never added code to check for that error. But

> if they do, what are their options? The motor still needs to be

> controlled and the status messages still need to be sent over whatever

> communication link the user needs. They're still going to need to fall

> back on some default in their code. Sending only 1 byte is the safest

> default, when you can't know. It's the most likely to allow projects to

> actually work, because if write() does block, the latency will be the

> smallest possible.

I'm trying to imagine the condition written by the user to check if the transmission blocks or not. I guess that the majority of use cases are like the following:

String position = readMotorPosition(...);

/* If write will not block then send position otherwise skip sending */

if (stream.availableForWrite() >= position.length()) {

stream.write(position);

}

Now let's suppose we want to be on the safe side and that we set the default of availableForWrite() to 1. In the example above the transmission will never happen anyway, unless the message length is exactly 1, but in this case returning 1 is not the safest possible value because it will quickly lead to blocking. I guess that to really be on the safe side we should return 0 because, since the availableForWrite is not implemented, we don't know how many bytes we can write without blocking and even 1 could block.

On the other hand, if we set the default of availableForWrite() to -1 (i.e. the maximum possible unsigned integer) we get the opposite result: the transmission will always happen and will likely block, but in this case we have the chance to check if the function is implemented, something like:

if (stream.availableForWrite() == -1) {

[...abort in some graceful way...]

}

String position = readMotorPosition(...);

/* If write will block then skip sending and retry later */

if (stream.availableForWrite() >= position.length()) {

stream.write(position);

}

Now I hardly imagine someone that implements the [...abort in some graceful way...] part (at least not me :-)), I'd better write something that skips transmission like:

String position = readMotorPosition(...);

/* If write will block then skip sending */

int avail = stream.availableForWrite();

if (avail != -1 && avail >= position.length()) {

stream.write(position);

}

that is basically equivalent to the first hypotesys (when availableForWrite returns 0).

Does this analisys sounds correct? Is there something that I'm missing?

C

[Bill Perry]

On 07/16/2014 07:02 AM, Cristian Maglie wrote:

I'm trying to imagine the condition written by the user to check if the transmission blocks or not. I guess that the majority of use cases are like the following:

String position = readMotorPosition(...);

/* If write will not block then send position otherwise skip sending */

if (stream.availableForWrite() >= position.length()) {

stream.write(position);

}

Now let's suppose we want to be on the safe side and that we set the default of availableForWrite() to 1. In the example above the transmission will never happen anyway, unless the message length is exactly 1, but in this case returning 1 is not the safest possible value because it will quickly lead to blocking. I guess that to really be on the safe side we should return 0 because, since the availableForWrite is not implemented, we don't know how many bytes we can write without blocking and even 1 could block.

On the other hand, if we set the default of availableForWrite() to -1 (i.e. the maximum possible unsigned integer) we get the opposite result: the transmission will always happen and will likely block, but in this case we have the chance to check if the function is implemented, something like:

if (stream.availableForWrite() == -1) {

[...abort in some graceful way...]

}

String position = readMotorPosition(...);

/* If write will block then skip sending and retry later */

if (stream.availableForWrite() >= position.length()) {

stream.write(position);

}

Now I hardly imagine someone that implements the [...abort in some graceful way...] part (at least not me :-)), I'd better write something that skips transmission like:

String position = readMotorPosition(...);

/* If write will block then skip sending */

int avail = stream.availableForWrite();

if (avail != -1 && avail >= position.length()) {

stream.write(position);

}

that is basically equivalent to the first hypotesys (when availableForWrite returns 0).

Does this analisys sounds correct? Is there something that I'm missing?

I generally agree with the ideas & concepts within the analysis for those specific examples;
however, I think that all those examples are a too simplistic.
Fragmentation is not that simple/easy. I wish it could be that simple but it isn't.
You have to track the bytes sent and then update counts, pointers, etc... for the
subsequent transmissions and to know when you are finished with that message.
The bigger issue is that the Print class write() function does not have support for
the String class. print() has String support but not write().
If you call print() with a String, it is synchronous/blocking as all the Print class functions
are synchronous. To change that without breaking lots of existing code,
I think you have to go back to what I earlier proposed which was adding a way to set the
blocking/non blocking mode on read() and write().
That would allow all the Print calls to return when the buffers filled up
when "non blocking" mode be enabled vs the way it works today
where it doesn't return until all the data bytes have been pushed down into the
library - which might cause blocking delays.

In the absence of adding "blocking" and "non blocking" modes,
using non blocking i/o will require using write() and specifying how much data
to send. i.e. write(buf, len);
and len can be calculated based on how the fragmentation is done.


The issue I have with using 0 or 1 for a default return value is that it is "by design" it is creating
an API that is unreliable.
That is to say the API doesn't always tell the truth to the application.
Sometimes it lies.
Both 0 and 1 are valid values to return when the functionality is implemented,
but when they are defaults, the application
has no way of knowing if they are the real values or whether they are defaults (lies).
Although in reality it is 1 that is the "lie" as 0 can be made to work - more below.

When the API returns a "not supported" type of status, the application can determine
how to react. It has multiple choices:
- it could choose to use 1 for a minimum block should it occur
- it could pick something other that 1 knowing what its worst case tolerance for blocking is
(i.e. say 9600 baud is used, and a block of 10 ms is tolerable, then sending 9-10 bytes would be ok)
- it could choose to just send the entire message and not worry about the block issue
- it could "panic" and choose not to run at all.

Note, it is possible to use 0 as the default value to indicate that the "write() available" function
is not supported.
The way this could work is that the application could do call to to availableForWrite() *before* it
sends any data. If it gets 0 then it knows that availableForWrite() is not implemented and it should
not be used.
The code then falls back into the same choices as mentioned above where it can now
choose how to handle the non blocking when the "available" function is not supported.
Code that does not  make this up front check and attempts to use availableForWrite(),
would get a zero and would never transmit anything,
which might not be a bad thing as it ensures a hard error when the functionality is not supported
vs creating something that kind of sort of works that becomes hard to diagnose why it has issues.
Essentially the 0 is indicating the maximum number of bytes that can be written where there is a guarantee
of no blocking. When the function isn't supported there is no guarantee of non-blocking so the value is zero.
It doesn't seem quite as clean as having a true status code, but it could work.


The key thing to keep in mind is that non blocking i/o and fragmentation is not a quick & easy thing to add
to a sketch.
It requires tracking pointers, counts, etc...

--- bill

[Rob Tillaart]

The funny thing in this discussion is that one will know compile time if the function is supported, 

so doing runtime tests to see if it is supported makes imho less sense.

If the device supports availableForWrite() one can use it as supposed.

if it is not supported you just do not use it. and create a different design.

[Bill Perry]

Can you elaborate further?
On how to add a new virtual function to Stream/Print that may or may not
be implemented down in the device library.

Were you assuming that it is not part of the Stream/Print API but just a
function in the individual device libraries?
I thought we had decided we wanted it to be in Stream/Print to ensure
that all libraries that implemented the same way.

The issue is not requiring the new function in the device library for backward compatibility with many
existing libraries that use the Stream/Print class.

If you don't have a default virtual function in Stream/Print,
then the compile of the older library code fails since they don't have this new function.

If you have a default function, then the compile & link works, but the sketch now needs
a way to detect if the underlying function down in the device library implemented it.

What is your solution?

--- bill

[Rob Tillaart]

> What is your solution?

pragmatically - I would do a test with the device library. I often do just to learn if the device and the driver lib behaves as expected or has anomalies.

[Bill Perry]

On 07/16/2014 03:48 PM, Rob Tillaart wrote:

> What is your solution?

pragmatically - I would do a test with the device library. I often do just to learn if the device and the driver lib behaves as expected or has anomalies.

???
But that is just a learning exercise to see how the device library behaves
which is outside the scope of this issue.

What is your solution for handling the issues I brought up earlier?

You stated that it could be determined a compile time.
If so, how do you do it and
how do you add a new function to the Stream/Print API without breaking
the the existing libraries?

You received this message because you are subscribed to a topic in the Google Groups "Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/a/arduino.cc/d/topic/developers/ls3hkviFYM4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to developers+...@arduino.cc.

[Paul Stoffregen]

On 07/16/2014 12:27 PM, Bill Perry wrote:

Note, it is possible to use 0 as the default value to indicate that the "write() available" function
is not supported.
The way this could work is that the application could do call to to availableForWrite() *before* it
sends any data. If it gets 0 then it knows that availableForWrite() is not implemented and it should
not be used.

My current implementation on Teensy, which I'm planning to use as the model for a pull request to Arduino, does in fact return zero when the HardwareSerial transmit buffer is full.  I believe this is correct behavior, if it will later return a positive integer when space becomes available in the buffer.

If that's not correct, please explain?  I really want to get this right for end users.

Regarding the rest of this lengthy thread, it seems the practical result of non-consensus, at least in the short-term, is probably going to be omitting this function from the Print/Stream class definition.  Perhaps this lengthy but well-intentioned argument is ultimately counterproductive?  It seems to be only reducing the likelihood of defining a virtual function, with any default value!

[Cristian Maglie]

Hi Paul,

In data giovedì 17 luglio 2014 15:37:05, Paul Stoffregen ha scritto:
> My current implementation on Teensy, which I'm planning to use as the
> model for a pull request to Arduino, does in fact return zero when the
> HardwareSerial transmit buffer is full. I believe this is correct
> behavior, if it will later return a positive integer when space becomes
> available in the buffer.
>
> If that's not correct, please explain? I really want to get this right
> for end users.

It is the correct behaviour, fortunately we all agree on that :-)

Maybe my last email was not so clear: I was analyzing the consequences of a
*default function* defined in *Print class* that returns 1, as you suggested on
your last email, and my conclusions, for now, is that 1 is wrong and it should
be 0.

> Regarding the rest of this lengthy thread, it seems the practical result
> of non-consensus, at least in the short-term, is probably going to be
> omitting this function from the Print/Stream class definition. Perhaps
> this lengthy but well-intentioned argument is ultimately
> counterproductive? It seems to be only reducing the likelihood of
> defining a virtual function, with any default value!

Do you think? This thread is long indeed, but it's not (yet) stuck. Of course
it depends on the wish from everyone to relax their position to reach an
agreement, but I still see some progress.

On the other hand I don't wan't to force you to wait for a decision on this
particular aspect so, as I already said, go ahead, and post your pull request
without the "Print/Stream" definition (that can be eventually added later).

C

[Bill Perry]

On 07/17/2014 08:37 AM, Paul Stoffregen wrote:

On 07/16/2014 12:27 PM, Bill Perry wrote:

Note, it is possible to use 0 as the default value to indicate that the "write() available" function
is not supported.
The way this could work is that the application could do call to to availableForWrite() *before* it
sends any data. If it gets 0 then it knows that availableForWrite() is not implemented and it should
not be used.

My current implementation on Teensy, which I'm planning to use as the model for a pull request to Arduino, does in fact return zero when the HardwareSerial transmit buffer is full.  I believe this is correct behavior, if it will later return a positive integer when space becomes available in the buffer.

If that's not correct, please explain?  I really want to get this right for end users.

That snippet of mine above was from a larger explanation of *also* using 0 to indicate that
availableForWrite() is not supported not reserving it for that indication.

I agree with that behavior for the availableForWrite() function when it is implemented.
A non zero return value indicates the indicates the maximum number of bytes that can be
sent to the device library using write() and be guaranteed of non blocking.
A return value of 0 by availableForWrite() means an immediate call to write() not guaranteed to be non blocking.
(i.e. it may block for some period of time)


The discussion is currently on what should the default value be when the "available" function
in the device library is not implemented.

My view is that a return value of 1 is not a good choice for a default function when avalableForWrite()
is not implemented in the device library.
And the above snippet was part of a longer explanation of how
0 could also be used as the return value by the default function when the availableForWrite()
functionality is not implemented by the device library.
0 is still a valid value when the functionality is implemented to indicate that blocking
may occur on the next write().

What I was also saying is that when 0 is used to indicate that availableForWrite()
is not implemented,
the way the application can tell if the availableForWrite() is implmented or not
is to call availableForWrite() once up front *before* it has ever sent any data to the device library.
(probably in setup())
At that point, if the availableForWrite() function is implemented in the device library,
the return value will be the maximum amount of data bytes than can be sent without blocking;
a return value of 0 would indicate that the function is not implemented or that there is always a possibility
of blocking.
The application can then take the appropriate action and decide how to deal with
no non-blocking support.
There are cases where avalableForWrite() might be implemented and yet blocking
will always occur - SoftwareSerial might be one example.

Since most of my original response about the choices and how this would work were not
included,  I'll  re-explain it (hopefully better this time).

Essentially the return value of availableForWrite() would *always* be indicating the maximum number of bytes that
can be sent to the device library using write() where there is a guarantee of no blocking.
This keeps the return value simple and consistent regardless of whether the function is implemented in the device library
and the API never lies to the application like a default return value of 1 does.
Also the return value is always this byte count, and not sometimes a count and sometimes a status.
When the function isn't supported by the device library, there is no way to know how many bytes can be sent using write()
without blocking, so the default function returns a value is zero.
Zero would seem to be the correct choice since there is no way to know how many bytes can be
written before blocking may occur so there is always the possibility that write() may block.

I believe using a default value of 0 would make more sense than a value of 1 since a value
of 1 is lying to the application and more importantly the application loses the ability
to tell if the availableForWrite() function is implemented.
A return value of zero also ensures that overly simple code that didn't check up front for non blocking support,
hard fails when ported to a device library that doesn't support it. This seems much better than if the default returns
1 and then the resulting blocking creates a situation that kind of sort of works, and is hard to debug.

Doing a check up front could also be useful to allow the messaging &  fragmentation code
to know its maximum message size. It could potentially simplify messaging code
(similar to the kind of code that Christian posted earlier),
if it knew the device library buffer space was large enough to hold the full messages.
If it wasn't large enough, it could potentially shrink the messages, deal with the potential for
blocking, or even hard fail.



In terms of how to handle a lack of non blocking support,
the application has  multiple choices:
- it could choose to fragment the message into 1 byte write() for a minimum block should it occur

- it could pick something other that 1 knowing what its worst case tolerance for blocking is
(i.e. say 9600 baud is used, and a block of 10 ms is tolerable, then sending 9-10 bytes would be ok)

- it could choose to just send the entire message and not worry about the blocking issue
- it could "panic" / hard-fault and choose not to run at all.


Summary:

Using a default return value of 0 seems quite useful:
- The API never lies to the application as is the case with a default return value 1
- The application can still tell whether non blocking io is supported.
- It avoids having to create a new error status value to indicate that the function isn't supported
- overly simplistic applications will hard fail when ported to new i/o libraries vs potentially misbehave


After some additional thinking about it, I think I'm now leaning twards using a default
return value of 0 vs defining an error status value because when using an error status value,
the overly simplistic code will not hard fail but rather always think that there is lots of room
since the error status, will actually be a very large number.

Using zero for the default seems pretty clean and simple.


--- bill

Regarding the rest of this lengthy thread, it seems the practical result of non-consensus, at least in the short-term, is probably going to be omitting this function from the Print/Stream class definition.  Perhaps this lengthy but well-intentioned argument is ultimately counterproductive?  It seems to be only reducing the likelihood of defining a virtual function, with any default value!

[Matthijs Kooijman]

Hey Bill,

> There are cases where avalableForWrite() might be implemented and yet blocking
> will always occur - SoftwareSerial might be one example.

Good point!

This (and the rest of your well-written mail, Thanks) actually made me
think a bit more, why not implement a second function writeBufferSize()
(in addition to the proposed writeBufferFree())? I imagine that knowing
the buffer size can be useful for applications that want to do advanced
stuff - if they're waiting for space to free up in the buffer so they
can send a full message, it would be good if they can tell if they won't
indefinitely because the buffer simply isn't big enough.

Having this second function also simplifies the backward compatibility
issue: both functions can default to returning 0, which indicates to the
application that no buffer happens and everything blocks (which I think
might even be _correct_ in a lot of custom Print applications).

It still does not unambiguously tell applications wether these two
functions are implemented or not, but I think the "no buffer" and "not
implemented" cases can be handled in the same way usually. If an
application needs advanced blocking behaviour, they should make sure
that all Print implementations implement these methods (IMO the default
implementations are _just_ for existing implementations, new
implementations should be encouraged to implement their own versions,
even if they would also just return the same value as the defaults, so
we could perhaps at some point in the future remove the default
implementations).

Bill argues that you can obtain the same information from aa single
writeBufferFree() function, by calling it once at startup. However,
given we're talking about generic libraries here, that take an arbitrary
Print instance to work with, there is really no way for those libraries
to be sure that the Print instance is in fact freshly initialize - for
all they know the same Print instance is passed to multiple libraries.

Hence, having an explicit writeBufferSize seems to make sense?

Gr.

Matthijs

[Paul Stoffregen]

On 07/17/2014 11:08 AM, Cristian Maglie wrote:
> On the other hand I don't wan't to force you to wait for a decision on
> this particular aspect so, as I already said, go ahead, and post your
> pull request without the "Print/Stream" definition (that can be
> eventually added later). C

Done.

https://github.com/arduino/Arduino/pull/2193

[Bill Perry]

My only question is on the actual virtual / non-virtual function declaration in HardwareSerial.h

I commented further on github.

--- bill

[morgan]

>> My belief is event callbacks, called from main program context, are by far the most novice-friendly way to provide concurrency.

Paul, have you considered the actor model ? it's harder to implement but seems pretty nice - xcore has a language that implements the main ideas quite nicely: https://www.xmos.com/support/tools/intro


Also maybe this could offer arduino users/learners a tool that could also serve them in building hard real-time systems(if they use xmos's chips) - which might be valuable to them.