I2C - Wire buffer size

[Rob Tillaart]

HI,

I'm using I2C quite a lot (especially EEPROM 24LC256/512) and it works quite well except one thing: the buffer size 

From Wire.h

[code]

#define BUFFER_LENGTH 32

[/code]

The 24LC256/512 have a page size of 64 bytes. A page is the maximum block that can be written to EEPROM in one write after which the 24LCxxx (and probably others too) have a 5 millisecond latency.

So if I want to write a page to EEPROM I currently need 3 writes as the WIRE TXbuffer needs 2 bytes for the address, leaving 30 bytes for data. 3 writes means 15 millis latency in total.

if the BUFFER_LENGTH can be upgraded to 34 only 2 writes are needed, so only a 10 millis latency in total. That would be an optimal trade memory for speed imho.

#define BUFFER_LENGTH 34

Comments, remarks, opinions?

Regards,

Rob

[Todd Krein]

Rob,

                It’s a trade-off. The AVRs have very limited RAM, and this buffer is essentially statically defined. Most folks, I think, use I2C to send very small messages, and so the extra is wasted. I have wanted to make the default smaller, since I get more people asking me to streamline it than expand it…

 

                On the other hand, two extra bytes are unlikely to break the bank.

 

                This is a perfect example of a case where being able to pass #DEFINES to a library would be a huge win.

 

                Cheers,

                Todd

--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[Rob Tillaart]

Hi Todd,

I understand the need to make the buffer smaller. If I reverse my reasoning, and accept 3 writes as minimum the buffer size can be reduced to 24 bytes for me without extra delay.

I like the idea of passing a parameter for the buffer size (not per se as a #define) and if possible I would prefer to define the size for TX and RX separately for optimal tuning!.

Rob

[Paul Stoffregen]

On 11/13/2013 09:56 AM, Todd Krein wrote:

I have wanted to make the default smaller,

Did you know the Wire library allocates not 1, but 5 buffers?  Hard to believe, but easy to verify.  Two are in Wire.cpp and 3 more in twi.c.  All 5 are 32 bytes each.

The Due version allocates 3 buffers.  My implementation for Teensy3 allocates 2 buffers.

It's impossible to get it down to just 1 buffer and still be compatible with the Wire API.  I tried this in my first implementation, but it turned out there are some very strangely coded sketches that depend on being able to use beginTransmission and write, then they do a read sequence, and after reading, they use endTransmission to send the data they previously wrote.  Those cases do exist "in the wild".  Using one 1 buffer can't work with them.

There might even be some extremely strange multi-master cases that need 3 buffers, where the sketch does that strange queue data, read, then write sequence while another I2C master sends or receives, causing the slave mode handlers to be called, while the main sketch still has both buffers in use?  My impression, admittedly from only a quick reading of the code, is Due's version appears to be designed to handle that case.

I have a hard time imagining any case where 5 buffers are truly needed in the AVR code.

Anyone who really, truly cares about wasted RAM on AVR, when talking about the Wire library, should focus first on those redundant buffers.

[Rob Tillaart]

Thanks Paul for this insight of the 5 buffers...,

So maybe we need a complete rewrite of the Wire library to use less buffers and name it I2C ?

[Todd Krein]

It sounds like, as Paul has done, I’ve spent a lot of time digging through the Wire library. (There are a number of bugs in the statemachine…)

 

Let me clean up what I have and I’ll submit it…

[Paul Stoffregen]

On 11/13/2013 10:56 AM, Rob Tillaart wrote:

Thanks Paul for this insight of the 5 buffers...,

So maybe we need a complete rewrite of the Wire library to use less buffers

Certainly Wire could use some pretty substantial refactoring, or something?

and name it I2C ?

Don't get me started again on the need for compatibility!!!!!11

Wire is probably the most widely used of all Arduino libraries.  Honestly, the API is a mess, that countless sketches depend upon.  Rare are software developers who could work on such code and resist the urge to improve the API with incompatible changes.

[Todd Krein]

Paul,     

                What is your concern WRT compatibility?                              (Just kidding)

 

                I’m happy to work on that, unless you have it done already… The existing API is, in my opinion, both hard to use, and really hard to code efficiently.

 

From: Paul Stoffregen [mailto:pa...@pjrc.com]

Sent: Wednesday, November 13, 2013 11:05 AM
To: Rob Tillaart
Cc: Arduino Developers

[Nils Springob]

I have an additional wish for the I2C rewrite: The buffers should be allocated dynamically:
void begin (int buffersize = I2C_DEFAULT_BUFFER_SIZE)
There is no panalty because dynamically allocatated RAM has the same size in memory as
statically allocated one (except the 2 bytes for the pointer).

--
Nils Springob
Am Denkmal 8, 52223 Stolberg
Tel: +49-2402-997-8391
Mobil: +49-163-6556110

[Todd Krein]

What do you see as the benefit of dynamic allocation?

[Victor Aprea]

Imho a test suite / verification approach is needed for something as fundamental as the I2C implementation for Arduino... has any work been done on that front?

That is to say,  for a chance to be accepted, it must be demonstrates to work on some subset of real, agreed upon hardware and use-cases.

Regards,
Vic

p.s. I'm desperately hoping to not open a whole other can of worms here.

[Nils Springob]

It would be possible to define the buffersize after the library was compiled... (no need for #define BUFFERSIZE XXXX)
Normal users just call Wire.begin() to get the standard buffer size,
advanced users call Wire.begin(64) to get a special buffer size...

[Matthijs Kooijman]

Hey Paul,

On Wed, Nov 13, 2013 at 11:04:42AM -0800, Paul Stoffregen wrote:
> On 11/13/2013 10:56 AM, Rob Tillaart wrote:
> >Thanks Paul for this insight of the 5 buffers...,
> >
> >So maybe we need a complete rewrite of the Wire library to use
> >less buffers
>
> Certainly Wire could use some pretty substantial refactoring, or something?
>
> >and name it I2C ?
>
> Don't get me started again on the need for compatibility!!!!!11

So, what's your opinion here, then? Stay with Wire and live with the
limitations of the API, or do you support the idea of creating a new
library with a better API (and keep Wire around for older sketches)? Or
perhaps you are suggesting that the Wire API can be enhanced with
sufficient effort while keeping backward compatibility too?

I'm not sure which would be better (haven't dug into the Wire API), I'm
just wondering what your idea is?

Gr.

Matthijs

[Todd Krein]

I know you didn't ask me, but having combed through the Wire library, there are some aspects of the existing API that severely limit the speed at which the interface can be run. I'd second Paul's suggestion that we add a new I2C library with a better API.

-----Original Message-----
From: Matthijs Kooijman [mailto:matt...@stdin.nl]
Sent: Wednesday, November 13, 2013 11:35 AM
To: Paul Stoffregen

[Paul Stoffregen]

On 11/13/2013 11:35 AM, Matthijs Kooijman wrote:
> Hey Paul,
>
> On Wed, Nov 13, 2013 at 11:04:42AM -0800, Paul Stoffregen wrote:
>> On 11/13/2013 10:56 AM, Rob Tillaart wrote:
>>> Thanks Paul for this insight of the 5 buffers...,
>>>
>>> So maybe we need a complete rewrite of the Wire library to use
>>> less buffers
>> Certainly Wire could use some pretty substantial refactoring, or something?
>>
>>> and name it I2C ?
>> Don't get me started again on the need for compatibility!!!!!11
> So, what's your opinion here, then? Stay with Wire and live with the
> limitations of the API,

Arduino 1.0 promised this API would remain stable. Wire's awkward API
must be maintained. The time to change it was 2+ years ago. Some
additions, like support for repeated start, were made. But sadly, we're
now stuck with a legacy of Wire's API.

> or do you support the idea of creating a new
> library with a better API (and keep Wire around for older sketches)? Or
> perhaps you are suggesting that the Wire API can be enhanced with
> sufficient effort while keeping backward compatibility too?

At least a couple alternate I2C libraries have gained widespread use.
The one I've seen on several projects is a master-only implementation
with an API for the register-based addressing that most (but not all)
I2C chips implement on top of the protocol.

It may be possible to extend or improve the API while still maintaining
backwards compatibility. A non-blocking endTransmission() with some way
to query for completion/status might be nice.

But some of the really desirable changes, like disallowing a read
sequence in the middle of a write sequence, just can't be done. Far too
many programs depend on the Wire library's quirks.