Made Ethernet library non Blocking

[Davy Landman]

Hi Guys,

c.maglie suggested I should also discuss a proposed contribution to the Arduino Library here.

originally I opened a pull request at github: https://github.com/arduino/Arduino/pull/1240 (so the code you find the code there, look at commit 2ac4e2f the other commit normalizes the tab/space usage in the library)

What did I do?

I've extended the Ethernet library to make it possible to work with it in a non blocking way, which makes it easier to combine the library with handeling other task (handeling buttons, updating a screen etc) while a DHCP lease happens, or a packet is sent.

I've added non blocking variants for:

• Dhcp::*
• Dns::getHostbyName
• Ethernet::begin (DHCP version)
• Ethernet::maintain
• EthernetClient::connect(*)

I welcome comments on the api-style, not really glad with all the names, but what would you guys think? I also need to think about a non blocking send, since it now loops until enough memory is available, I could make a variant which actually returns the bytes it could send, but I still have to think if this is the best way to do this.

There are 2 commits in this pull request, the Ethernet library was a mix of 3 different indentation styles, so I looked at the other parts and made commit a2ca62d to first only fix the formatting of the code. Then there is a new commit which actually changes the code, commit 2ac4e2f . This is actually an import from the work I started in DavyLandman/NBEthernet where I found out that the existing Ethernet library was already half non blocking.

So when reviewing, make sure to select just the correct commit (2ac4e2f), else in the github summary will claim I changed almost everything..

Impact on the size is quite minimal (for example DnsWebClient ):

old: Binary sketch size: 13,474 bytes (of a 30,720 byte maximum)
new: Binary sketch size: 14,980 bytes (of a 30,720 byte maximum)

The increase is primarily due to keeping a blocking variant.

I've tried to add "documentation" to the .h files where I added a new method. But this here is an example, which uses all the new asynchronous methods. A user can mix them:

EthernetClient client;
static uint32_t next_connect;

void setup() {
    // start the serial library:
    Serial.begin(9600);
    Serial.println("Initializing Ethernet controller");
    Ethernet.initialize(mac);
    int result = 0;
    do {
        result = Ethernet.initialized();
    } while (result == 0);

    if (result == 2) {
        Serial.println("Failed to configure Ethernet using DHCP");
        while (true) ;
    }
    Serial.println("Got IP adress");
    Serial.println(Ethernet.localIP());
    next_connect = millis() + TIMEOUT;
}

static byte maintaining = 0; // DHCP maintain happening
static byte insession = 0; // a HTTP GET is in session 
static byte waiting = 0; // waiting for the connection to be established
static uint16_t bytes_read;

// crapy state machine, for real applications, use something better ;-)
void loop() {
    if (maintaining) {
        int result = Ethernet.maintainFinished();
        maintaining = result == 0;    
        if (!maintaining) {
            if (result == 1) {
                Serial.println("DHCP succeeded");
                Serial.println("Got IP adress");
                Serial.println(Ethernet.localIP());
            }
            if (result == 2) {
                Serial.println("DHCP failed");
            }
        }
        return;
    }
    else {
        maintaining = Ethernet.maintainNeeded() != DHCP_CHECK_NONE;
        if (maintaining) {
            Serial.println();
            Serial.println();
            Serial.println("DHCP timeout, getting new one");
            return;
        }
    }
    if (!insession) {
        if (!waiting && millis() >= next_connect) {
            next_connect += TIMEOUT;
            Serial.println("Starting a new connection");
            if (!client.initializeConnection("www.google.com", 80)) {
                Serial.println("initializing connection failed");
            }
            else {
                waiting = 1;
            }
        }
        if (waiting && client.connectionInitialized()) {
            if (client.connectionInitialized() == 1) {
                Serial.println("connected");
                client.println("GET /arduino HTTP/1.0");
                client.println();
                waiting = 0;
                insession = 1;
                bytes_read = 0;
            }
            else {
                waiting = 0;
                Serial.println("something went wrong with getting a connection");
            }
        }
    }
    else {
        // we have a connection, and the GET has been sent.
        // lets consume the response
        if (client.available() && bytes_read < READLIMIT) {
            char c = client.read();
            Serial.print(c);
            bytes_read++;
        }
        else if (client.available() && bytes_read == READLIMIT) {
            client.flush();
        }

        if (!client.connected()) {
            Serial.println();
            Serial.println("disconnecting.");
            client.stop();
            insession = 0;
            waiting = 0;
        }
    }
}

Cheers,
Davy Landman

[Matthew Ford]

I think 'non-blocking' would be a big addition for Arduino.
I have non-blocking ADC code and non-blocking SoftwareSerial (9600 baud
only) in Atmel C at the moment
and will be looking at transferring these to Arduino when I finish the
current ATtiny861 project and try and move it to Arduino.

Having said that. From the inexperienced programmer's point of view,
blocking is simpler to understand and program and
was the right choice for Arduino's first pass.

So non-blocking implementations need to be very simple and clean and
appealing to use.
matthew

On 15/03/2013 12:24 AM, Davy Landman wrote:
>
> Hi Guys,
>
> c.maglie suggested I should also discuss a proposed contribution to the
> Arduino Library here.
>
> originally I opened a pull request at github:
> https://github.com/arduino/Arduino/pull/1240 (so the code you find the code
> there, look at commit 2ac4e2f the other commit normalizes the tab/space
> usage in the library)
>
> What did I do?
>
> I've extended the Ethernet library to make it possible to work with it in a
> non blocking way, which makes it easier to combine the library with
> handeling other task (handeling buttons, updating a screen etc) while a
> DHCP lease happens, or a packet is sent.
>
> I've added non blocking variants for:
>

> - Dhcp::*
> - Dns::getHostbyName
> - Ethernet::begin (DHCP version)
> - Ethernet::maintain
> - EthernetClient::connect(*)

> static byte maintaining = 0; // DHCP maintain happeningstatic byte insession = 0; // a HTTP GET is in session static byte waiting = 0; // waiting for the connection to be establishedstatic uint16_t bytes_read;
> // crapy state machine, for real applications, use something better ;-)void loop() {

[Paul Stoffregen]

On 03/14/2013 04:29 PM, Matthew Ford wrote:
> I think 'non-blocking' would be a big addition for Arduino.
> I have non-blocking ADC code and non-blocking SoftwareSerial (9600
> baud only)

I wrote a different software serial that does not block. Well, at least
not like SoftwareSerial does.

http://www.pjrc.com/teensy/td_libs_AltSoftSerial.html

As for non-blocking, there really should be a function similar to
available() that tells how much can be written to a stream without
blocking. This was discussed over a year ago, and there seemed to be
some consensus on the idea (but not the function name). Nobody ever
seems to have implemented it.

[Matthew Ford]

Your approach appears (without looking at the code in detail) similar to
mine.
I consume one timer, which I also use for the millis() fn, so no
additional timer is taken.
I limited my serial baud rate to 9600 too keep the tx/rx load to <10%
of the cpu (at 8Mhz) so that other functions were not starved.
matthew

[Davy Landman]

Interesting discussion, however it seems unrelated to the Ethernet non blocking code?

Regarding the non-blocking usage as a simple user, I agree it is more complex, so we should keep the blocking overloads too perhaps?

However, might want to have an option to disable those, just to decrease size?

[Knight, Dave]

IMHO, all "production" libraries should be non-blocking, to the greatest extent possible.  This would make it far easier to build reliable, high-function applications.  Some of the Arduino multi-tasking capabilities being worked on should help in this regard as well. 

Currently, "non-blocking" Arduino sketches, that appear to be doing number of things in parallel, are pretty easy to write if you have a good grasp finite state machines.

For example, it is easy to devise non-blocking functions to: 1) read a number of temps from thermistors on the ADC pins,  2) maintain current GPS info (e.g. from a Skylab SKM53, 9600 baud) using the TinyGPS and Time libraries, 3) update various regions of a 4x20 LCD as necessary to reflect date/time, operating conditions, etc. and "dispatch" them every second or so from the loop() function.

However this gets messy if you want to use libraries with blocking functions concurrently with time-dependent operations such as serial input.  For example, when using the UTFT graphical display library (whose output functions block) and the TinyGPS libraries (which uses serial input and is thus somewhat time-dependent), the display update must be split up into many short UTFT "print" operations to minimize block-time**, each with its own "state", such that they are interspersed with the GPS serial input reads, otherwise GPS data will be lost due to serial input buffer overruns - to the point of seldom, if ever, receiving a complete GPS "sentence" that passes the checksum test!

** Arduino serial I/O "runs in background", to a point.  Software serial has a single 64 char input buffer, and 9600 baud is about 960 chars/sec, suggesting that if something in loop() blocks for more than 50ms while serial input is being received, there will be a buffer overrun.  For hardware serial, which has a pair of 128 char input buffers, there is more leeway, like maybe 150ms.