File base class in the core library?

[Paul Stoffregen]

Perhaps the time has come to consider a File base class in Arduino's
core libraries?  Several such classes already exist, like Client,
Server, UDP, Stream, and I believe it's pretty safe to say they've been
very successful at allowing libraries that publish these sorts of
resources to be used interchangeably with sketches and other libs which
need them.

Currently two libraries published by Arduino define File classes: SD and
Bridge.  3rd party libraries also allow accessing files. Andrew's USB
Host Shield v3 provides access to files on USB memory sticks.  My
SerialFlash lib gives file-like access to serial flash chips.  Bill's
SdFat library also gives access to SD cards.  I believe the ESP folks
are also providing a small filesystem with the remaining memory in their
devices, though I'm not very familiar with the details.  We can expect
more libraries to provide access to files in the future, either directly
to media or over communication protocols like NFS, SMB, MTP, etc.

The long term problem is libraries & sketches can't be written to *use*
files from any library which provides filesystem access. The result is
most file-accessing programs using File are locked to the SD library. 
Bridge uses a namespace to give an API very similar to SD, but you can't
even compile any program which includes both FileIO.h and SD.h, much
less write a library like a JPEG decoder which accepts a .JPG file from
either, not to mention 3rd party libs like USB Host Shield.

I know this is a large & difficult subject.  Previously the decision was
to use the namespace to solve a conflict.  But the Arduino community is
left without any way to create libraries that actually *use* files from
any source providing filesystem access. My hope is we can revisit the
"File" API and come up with a base class that allows File using
libraries to work with data from any File providing library.

[Massimo Banzi]

Paul 

Thanks for bringing this up, this is very important.

We do need this, badly.

I was thinking that we should probably have :

*  BlockDevice interface for the underlying stuff (we have Stream for the character interfaces we should have one for block devices)

* Filesystem (to based on SDCard?)

* File , unifying the different implementations.

It would be interesting to figure out how to make it somehow compatible with the filesystem class in ESP8266, we should aim to have more collaboration with that team and make sure there is not too much divergence in the APIs..

What do you think about this?

M  

--

Massimo Banzi

m.b...@arduino.cc

--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[Steve Weinrich]

Excellent notion.  Perhaps we should look at the STD C++ classes.  I seem to recall that they have classes for character I/O and for a filesystem.

[Andrew Kroll]

As Paul mentioned, my fs lib allows "normal" fileopts, just like normal, more-or-less.
https://github.com/felis/UHS30/tree/master/libraries/UHS_FS

--
You received this message because you are subscribed to the Google Groups "Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@arduino.cc.

--

Visit my github for awesome Arduino code @ https://github.com/xxxajk

[Vicne]

Hi,

I am somewhat familiar with the ESP filesystem, SPIFFS for SPI Flash FileSystem. 

SPIFFS was developed and is maintained independantly by Peter Andersson for more than 4 years and stable versions (currently 0.3.7) are regularly integrated in the ESP8266 Arduino project tree. 

Currently, there is no common base class for that implementation and the SD library for example, and conflicts on the "File" class pop up regularly (see this issue for example), and the "solution" is to use different namespaces so they can coexist, but implementations cannot be used interchangeably. 

However, SPIFFS version 0.4 is currently being rewritten from scratch, so now may be the perfect time to discuss potentially breaking changes with Peter. Questions on 0.4 are handled here.

Just my 2 cents

Vicne

[Thomas Roell]

Sorry to be so late to the party with this.

I'd suggest to do this properly this time.

The SD class mixes up File access with Directory access.  That improper layering does cost a lot of extra code, memory overhead and performance overhead. 

The mentioned ESP8266 FS.h wrapper solves the problem almost appropriately, by introducing a "Dir" object.

Then there is this File::name() operation. On a non-VFAT FAT12/FAT16/FAT32 system this is nice and dandy, because a filename has a reasonable length (13 bytes including the trailing '\0'.). With VFAT that goes up to 260 (I believe; there were some special cases that did not allow for the 255 character limit). But then again VFAT uses UNICODE, so the effective storage requirement might be even higher (assuming you'd use UTF-8 over unicode). So that API is a bad idea for anything but a non-VFAT file system. Yes you could be nitpicky and say the SFN name is used in the VFAT case, but that breaks if you want to move on to exFAT.

- Thomas

 

--
You received this message because you are subscribed to the Google Groups "Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@arduino.cc.

[Andrew Kroll]

Actual file system handling should be a total separate component from the actual file access.

You also don't need to do it as a class. It should be in plain C.

I just did it in my library in the usual generic way and as far as directories are concerned, I just use the Unix style of path and mount, with the actual mount point being managed by the disk label.

Go check to see how it is done in the examples in my most recent Library. The cool thing about it is that you can add other filesystems to  it because it is filesystem agnostic. For example if you wanted to be able to read iso9660 as in CD ROM, implementing the hook is very easy to do. I haven't had the opportunity to put in an SD card hook yet that would be on SPI but doing so would not be that difficult. However since you can just use a SD reader on USB I didn't think it would make any sense to do so...

[Thomas Roell]

Yeah, here another example:

https://github.com/GrumpyOldPizza/arduino-STM32L4/blob/master/cores/stm32l4/FS.h

https://github.com/GrumpyOldPizza/arduino-STM32L4/blob/master/cores/stm32l4/FS.cpp

https://github.com/GrumpyOldPizza/arduino-STM32L4/blob/master/system/STM32L4xx/Include/dosfs_api.h

So yes, there are more than one way to skin this cat.

My point however is that the structuring that SD.h introduced is rather bad, and it does not scale to other non-FAT file systems (with longer file names), nor does it scale space/performance wise. If we add a new abstract File/FileSystem class this should be addressed, so that it does not become an issue in the future.

ANSI-C/POSIX/WIN32 did solve this problem ages ago, so it stands to reason that their layering into file/directory/volume is reasonable.

- Thomas

[Massimo Banzi]

I would like to implement:

* a BlockDevice interface like on mbed (as a companion for Stream which is our "character divide “)

* FS from the ESP8266 port

* Move SDCard and File on top of these interfaces/classes

This will give us a fairly flexible way to implement a ton devices and filesystem (for example we can then use https://os.mbed.com/blog/entry/littlefs-high-integrity-embedded-fs/ )

Thomas: https://github.com/GrumpyOldPizza/RFAT looks very interesting too!

What do you guys think?

M

--

Massimo Banzi

m.b...@arduino.cc

To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[Adrian Godwin]

I'm not sure of the current state of the FAT/exFAT patent problem. Does it have any implications here ?
It would be a shame if devices made using Arduino were subject to attack by MS lawyers.

[Thomas Roell]

Massimo,

thanx for pointing out the mbed BlockDevice interface. Need to think a few things throu with that to make a full assessment. Right off the bat there are a few concerns as to whether it's usable directly:

- it mixes up higher level managed block devices (SDCARD, eMMC, UFS) with unmanaged block devices (SPI FLASH, embedded NOR FLASH ...)

- there is no ::sleep() API

- there is no ::commit() type API; normal managed block devices keep a write back cache; you want to buffer as much data as you can in this cache, but you need a way to flush it; for an SDCARD this would be the "STOP TRANSMISSION" mechanism.

- there is no way to query the "allocation unit size" of a managed block device (need to align efficient streaming writes)

- the ::erase() API is pointless for managed block device as specified; a operation that perhaps maps a erased data block to the virtual address might be interesting (similar to the SDCARD ERASE command). [please check on eMMC how they define ERASE / TRIM / DISCARD, and how UFS use UNMAP instead; attached a nice simple paper that explains the background]

- there is no handling of ejectable block devices ... especially no support to detect media change, as any API that would identify the device (serial number, device type ...) is missing

> Thomas: https://github.com/GrumpyOldPizza/RFAT looks very interesting too!

Kind of an interesting project. The more up to date code is with the other link where I renamed it into "dosfs" with the expectation to add exFAT at some point. There is a few neat things (where I am always toying around). Write speed vs. power consumption. This was originally partly done to have high speed data logging for a autonomous robot. But high-speed logging requires to keep a lot of data in some caches, which then again prevents the SDCARD from going into a low power mode. The other interesting part (and perhaps I need to be more vocal about that) is that RFAT/dosfs can be configured to be transaction safe. Meaning you can remove power at any point of time, and the FAT file system structure will not be corrupted. 

>  https://os.mbed.com/blog/entry/littlefs-high-integrity-embedded-fs/ 

I am not sure this is needed or desirable. There are 2 main points that the presentation makes. One is that "LittleFS" is "power resillient". As pointed out with the RFAT comments above, that is possible with a normal FAT file system as well. The other point it tries to make is that FAT has no wear leveling. That is kind of correct if you don't know what you are doing. However since the dawn of time (NOR FLASH wise) there had been block translation systems in place ("understanding the flash translation layer (ftl) specification" is a good starting point to read up on ...). Here an implementation of a almost identical concept (sorry, I hacked around; the code mixes the SPI NOR FLASH driver with the actual FTL, but it's something to look at):

https://github.com/GrumpyOldPizza/arduino-STM32L4/blob/master/system/STM32L4xx/Include/dosfs_sflash.h

https://github.com/GrumpyOldPizza/arduino-STM32L4/blob/master/system/STM32L4xx/Source/dosfs_sflash.c

Historically there used to be this discussion of FTL vs. a real flash file system. Looking around, the clear winner is FTL. SDCARDs embed a FTL. You cell phones are using either eMMC or UFS, which embeds an FTL. 

And then there is this nagging question, "how do I get my data off my embedded device ?". If you add USB/MSC (which is really SCSI block device access via USB), does your PC or Mac understand "LittleFS" ? Or does it understand FAT12/FAT16/FAT16/VFAT/exFAT ?

Some additional remarks with regards to storage technology. Right now IMHO SDCARD and NOR FLASH are the most interesting ones. Latter ones now as SPI, QuadSPI and OctSPI interfaces. However there are also NAND FLASH devices with SPI/QSPI interfaces now, which require a substantial different way of handling them. A FTL can abstract that and you end up with a managed block device again. If you go low level with a specialized flash file system, you need a much different approach than with NOR FLASH.

Memory consumption. I *think* with the current source base, dosfs (FAT12/FAT16/FAT32/VFAT, NOR FTL, excluding the sdcard/spiflash backends) the SRAM requirement is probably about 3 to 4 kB. That includes caches, transport buffers, volume information, FTL translation tables, scratch buffers  ... I suppose using clever tricks, that could be reduced at the expense of performance. But given the amount of SRAM available in a typical mid range Cortex-M0+ system, that seems a reasonable tradeof. 

- Thomas 

[Thomas Roell]

FAT12/FAT16/FAT32 never had a patent that was enforceable associated with them.

VFAT (long file names) expired in 2016.

exFAT is covered by a patent. exFAT is interesting for SDXC cards. *HOWEVER* it is possible to format a SDXC cards, 64GB and 128GB, properly with FAT32 without violating the SDCARD File System Specification (which defines the system and data areas; important for the FAT1/FAT2 layout).

One issue that is underlying there is the commercial aspect. Over time SDCARDs have migrated to higher capacities. It's by now almost impossible to buy a 4GB SDHC card. Mostly 16GB and 32GB cards are being sold. In a few years SDHC cards will probably disappear alltogether. 

IMHO exFAT does not add a lot of value, other than allowing for larger cluster sizes, which in turn allows for more efficient data streaming for video recording devices.

- Thomas

[Paul Stoffregen]

On 01/09/2018 05:07 AM, Massimo Banzi wrote:
> * FS from the ESP8266 port
>

> What do you guys think?

Does this mean you're envisioning both File and Dir base classes in
Arduino's core library, and implementing Dir in Arduino's SD & Bridge
libraries?  (presumably SD would maintain openNextFile for backwards
compatibility, but access to directories from generic file-using libs
through the base class virtual functions would require use of Dir)

Or maybe you're imagining Arduino keeping File as-is for the base class,
forever dedicating the File API to do double-duty for both access or
ordinary files and also directories in all libraries providing
filesystem access?  (presumably the ESP folks would keep Dir for
compatibility, but redundantly implement SD-like directory functionality
under File)

Personally, I would prefer to see both File & Dir in the core library,
where the File base class does *not* require isDirectory(),
openNextFile(), rewindDirectory() from the inherited classes.

[Paul Stoffregen]

I'm studying the esp8266 FS.h File & Dir classes.  Something I don't understand about the Dir API is how subdirectories are supposed to be handled.  Hopefully someone more familiar with ESP can chime in?  Or maybe this is just an artifact of no support at all for subdirectories in the only filesystem code actually using this API?

In Arduino's SD library, as you traverse the entries from a directory, isDirectory() gives you a boolean so you can know if you've found a regular file or subdirectory.  I believe at the very least isDirectory() or some sort of function to get the type of file would be needed if Arduino where to adopt Dir as a base class.

I'm also struggling to understand why esp2866's File class has a name() function.  Clearly this is needed if there is no Dir class and File is used for accessing directories.  Maybe they started from SD's File and this is merely a leftover which nobody really uses?  Or maybe ESP users do use File::name() for something?  Again, hoping someone familiar with ESP can comment....

[Vicne]

Hi,

from  https://github.com/esp8266/Arduino/blob/master/doc/filesystem.rst :

Behind the scenes, SPIFFS does not support directories, it just stores a "flat" list of files. But contrary to traditional filesystems, the slash character '/' is allowed in filenames, so the functions that deal with directory listing (e.g. openDir("/website")) basically just filter the filenames and keep the ones that start with the requested prefix (/website/). Practically speaking, that makes little difference though.

So there is no "Directory" object, and of course, there's no such thing as an "empty directory", you don't create or delete directories, etc. One can think of it as "auto-created" and "auto-deleted" directories, and in the end it's surprising how users can develop perfectly valid code while completely ignoring that directories just don't exist.

As for the reason why the SPIFFS API is the way it is, the question should probably directed to the its author Peter Andersson - https://github.com/pellepl/spiffs - because according to Github, SPIFFS predates the ESP8266 Arduino project by more than 2 years.

Vicne

[Develo]

Hi all,

I'm a maintainer of the esp8266 repo. From reading this thread, it seems to me that only polymorphism is being considered here, i.e.: a base class and derived classes representing specific file systems. Polymorphism is rather ill suited to systems that are memory constrained, as there is always some overhead involved. Polymorphism is well suited for cases where you have to have a container that is populated at runtime with objects that are of different types, and you need to iterate over them to handle them as though they were the same type. I think it is realistic to consider that in most use cases, if not all, the user app considers only a very reduced number of systems (e.g.: 1-3), and there is no need to have them in some super file system container.

I would like to ask whether a design based on templated classes has been considered? I.e.: a template class, with specializations for the specific filesystems, all implementing the same interface. This would allow an implementation with zero overhead in memory, better compile-time optimizations, and stricter compile-time type checking.

- Dev

[me-no-dev]

Hi all! I am someone that can help with ESP8266 and ESP32. SPIFFS does not really support subdirectories but It could be simulated through software. In ESP8266 the API was driven by the SPIFFS API and we did not want to touch SD (Ivan wrote the current FS layer). In ESP32 I decided to rewrite it all and make the API SD compatible for all file systems.

Now the biggest issue probably is that ESP8266 is now a quite mature platform with many users and examples that depend on the current API. Maybe a compatibility layer of some sort can be made for ESP8266 to support both schemes. Difference is not much.

[Paul Stoffregen]

I used to feel this way about the overhead of virtual functions, and even now I'd love to explore any idea that's truly more efficient.  So I'm hoping we can talk about the details of this template specialization idea?

Ultimately the goal is to end up with a File class similar to Stream, where any library that uses I/O can take a Stream class without anything specific to whether that Stream is HardwareSerial or SoftwareSerial or Client or anything else implementing Stream.  We really also need to be able to return a File (for example, from some library that manages groups of files, even provided by different sources/libs) and have the caller be able to use the returned File with any other library that reads or writes files.  I'm a little skeptical that can be done without dynamic polymorphism, but if there's a proposal, I'll certainly give it a try.

Arduino has traditionally had a hard limit against APIs which would require template syntax in sketches.  So even if it's efficient and meets all the other needs, I'm pretty sure they're never going to accept any proposal that would require functions in a sketch which uses the File class as input to look like this:


  template<class T>
  void logMyDataToSomeFile(&T myfile, byte mydata[], int count) {
    for (int i=0; i<count; i++) {
      myfile.println(String("b:") + mydata[i]);
    }
    myfile.flush();
  }


This really needs to be able to work (at least in sketches) like this:


   void logMyDataToSomeFile(File myfile, byte mydata[], int count) {
     // ....


I can't speak directly for Arduino, but I have followed this mail list and most google code + github issues about APIs since the very early days.  Many proposals have died with a soft rejection (usually lack of response, ambiguous answers, or just drowning in "bikeshedding"), but anything that would require template syntax into sketches has always received a very hard & definitive "No" answer.  Sometimes Arduino devs have asked whether the template stuff could be buried in preprocessor macros to make it look like regular functions.  But even if that were to happen, I'd imagine the compiler showing complex template-syntax errors when things go wrong could also be a deal breaker.

However, I really do identify with the nagging feeling that we're wasting precious RAM on vtables, plus a few extra CPU cycles to make commonly used function calls.  In the cases where the compiler can't optimize these away, it does have a cost.  But it also has a tremendous benefit.  Not long ago, I personally looked at almost anything with a cost as something to avoid.  Sometime I still do.  But now I like to think about value, whether or not something is worth its cost.  I believe the Print, Stream, Client, HardwareSerial, and other polymorphic base classes have shown their worth many times over, facilitating the Arduino ecosystem.

One other quick point I'd like to make is about the compiler optimizations in newer versions of gcc.  In many cases, the compiler does "devirtualization", completely eliminating the overhead a compile time.  Several factors can make difference, such as use of constexpr for constructors, use of keywords like "override", and whether link time optimization is used.  Long-term, as we all migrate to newer compilers, most programs using polymorphism will have this overhead automatically optimized away at compile time.

[Cristian Maglie]

Il 19/01/2018 19:27, Develo ha scritto:
> type. I think it is realistic to consider that in most use cases, if not
> all, the user app considers only a very reduced number of systems (e.g.:
> 1-3), and there is no need to have them in some super file system container.
> I would like to ask whether a design based on templated classes has been
> considered? I.e.: a template class, with specializations for the
> specific filesystems, all implementing the same interface. This would
> allow an implementation with zero overhead in memory, better
> compile-time optimizations, and stricter compile-time type checking.

Hi Develo,

Sounds interesting, may you provide an example of how such templated
class will look like?

--
Cristian Maglie <c.ma...@arduino.cc>

[Develo]

Hi,

I've (slowly) started to put together an outline of what a template-based design could look like. At this time, I'm aiming for similarity with the current object interfaces, but not compatibility. We can discuss compatibility later, as I want to make sure the fundamental design is unhindered by that.

Also, what I'm putting together is not off the top of my head, but rather the result of a discussion with a work colleague, who is somewhat of a C++ guru (I consider myself a C++ expert, but this guy knows things that no normal sane C++ programmer should know. Ever.).

Given a sufficiently good design, the implementation I'm aiming for should completely separate the concepts of file system, file, directory, and underlying storage (flash, SD card, heap, whatever). This is important, currently e.g.: a File is necessarily tied to a filesystem, so Files from different filesystems are not compatible objects.

About usage, I think we can agree that usage of the STL is rather simple, and that the onus of implementation is all hidden from the user. In other words, usage of e.g.: std::string, std::vector, std::list, etc is pretty simple from the user's point of view, independently of how the implementation looks like inside. I'm trying to follow similar guidelines. 

It may take some time, but I'll report back here once I have something to show.

 - Dev

[Cristian Maglie]

Hi Develo,

thanks but what I'm asking for is an example of how a templated class
works and looks like to see if it is suitable for Arduino. If you can
show me how a "templated class" work we can see if this is a good fit
for an Arduino API or not. Anyway, the fact that you need a C++ guru to
put it together it's not a good sign...

> About usage, I think we can agree that usage of the STL is rather
> simple, and that the onus of implementation is all hidden from the user.
> In other words, usage of e.g.: std::string, std::vector, std::list, etc
> is pretty simple from the user's point of view, independently of how the
> implementation looks like inside.

Let me stop you here immediately, because we do not agree: STL is not
simple and is not suitable for the typical Arduino user. As you may have
noticed, and someone else pointed out already, Arduino do not use any
template syntax in his user-visible API, STL is out of discussion here.
--
Cristian Maglie <c.ma...@arduino.cc>

[Steve Weinrich]

I suggest that you use composition instead of templates.  This is a very nice way to build large systems.  The short of this technique is to create small worker classes that do simple task(s).  Then, one creates a usable class delegating to various combinations of the worker classes.  Here is a contrived example:

// worker

class FileOpenUnix {

{

  public:

    int8_t open (/* some appropriate args */ );

};

// user

class FileUnix {

{

  public:

    int8_t open (/* args */ ) return mFileOpenUnix.open(/* args*/ ); }

 private:

  FileOpenUnix mFileOpenUnix;

};

// user

class FileLinux {

{

  public:

    int8_t open (/* args */ ) return mFileOpenUnix.open(/* args*/ ); }

 private:

  FileOpenUnix mFileOpenUnix;

};

One can see that with the correct worker classes one can create appropriate combinations to fit any system without using virtual methods.  I believe this might be a better fit for the Arduino.

--
You received this message because you are subscribed to the Google Groups "Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@arduino.cc.

[Thomas Roell]

Don't want to throw a hair into the discussion, but here a common use case:

File myFile;

void setup(void)

{

    myFile = <some files system class>.open("my_file.txt" ...)

}

So in a nutshell the abstract base class "File" needs a way to link back to it's "FileSystem". It seems that one can do that either via a vtable for the File class (i.e. a virtual base class), or store a pointer in "File" to the "FileSystem". In the latter scenario "FileSystem" needs to be a virtual base class.

I cannot see any way how you could do the same thing with templates (and allow the upper example to work).

The downside of using virtual base classes is that if the File class exposes a "seek()" operation, this code will be sucked in, no matter whether the final application really uses "seek()" or not.

- Thomas

 

[Paul Stoffregen]

Perhaps you could share more detail *before* investing a lot of time and effort?

[Develo]

Hi Cristian,

do you mean end users of the Arduino core, or providers of 3rd party frameworks? I can agree with you about the first: the typical Arduino end user shouldn't have to know or care about template use. However, I don't agree about the second. If someone decides to start providing code to a possibly wide user base, then he should know what he's doing, and that includes knowing about design concepts and the language itself. This is especially true for 3rd party libs that are commercial. Even then, there are ways, as I show below.

Here, we're talking about designing a templated base that will provide usability of specific types to the user. Authors of other specific types should be able to make use of this. We're not talking about pushing templated usage to the user, which is completely different. 

As example, use of STL for specific types is as simple as any other C++ objects. Consider the following:
You (Arduino) want to provide specific containers to your user base. Let's say a vector of ints and a vector of floats. You could provide this:

using intvector = std::vector<int>;
using floatvector = std::vector<float>;

Then the user does this:

intvector myInts;
myInts.push_back(1);
myInts.push_back(2);


floatvector myFloats;
myFloats.push_back(3.);
myFloats.push_back(4.);

There are no templates on the user side, and your implementation was 2 lines of code.

Let's say that you don't like the STL object interface, i.e.: method names. You could do composition, e.g.: provide something like this:

class intvector
{
 private:
  std::vector<int> mVector;
 public:
  void append(int val) {mVector.push_back(val);}
  ...
};

Then the user does this:

intvector myInts;
myInts.append(1);
myInts.append(2);

Again, no template usage on the user side. With such composition, the templated part is pushed even further away from the user. You can also decide what to expose from std::vector, as well as what to hide.

Let's say now that some 3rd party lib provider wants to offer his end user base a way to print Arduino-provided vectors out to Serial. He could implement and offer this:

template <typename vectorType>
void printVector(const vectorType &vec)
{
  for(auto elem : vec)
  {
    Serial.print(elem);
    Serial.print(' ');
  }
}

Alternatively, if templates are "not simple" for this 3rd party lib provider, he can use overloads:

void printVector(const intvector &vec)
{
  for(auto elem : vec)
  {
    Serial.print(elem);
    Serial.print(' ');
  }
}
void printVector(const floatvector &vec)
{
  for(auto elem : vec)
  {
    Serial.print(elem);
    Serial.print(' ');
  }
}

In either case, the user then does this:

intvector myInts;
myInts.push_back(1);
myInts.push_back(2);

floatvector myFloats;
myFloats.push_back(3.);
myFloats.push_back(4.);

printVector(myInts);
printVector(myFloats);

It's a templated design base, but there is no end-user template usage. Also, no inheritance, no vtables, no pointers to some base class, no object slicing, no overhead.

Now, let's say that somebody comes up with a brand new vector type: a vector of Strings! The extension for the templated version above would look like this:

using stringvector = std::vector<String>;

That's it. 

For the overloaded version, you'd need in addition this:

void printVector(const stringvector &vec)

{

  for(auto elem : vec)

  {

    Serial.print(elem);

    Serial.print(' ');

  }

}

For the filesystem design, the approach is similar. I was going to put an example here, but I decided to not jump the gun yet. I would like to try and show something that, while maybe only a skeleton and very incomplete, can at least serve as a starting point.

About the C++ guru, if you're going to implement a good design, you think twice about that design, so that the end users have to think only half about its usage.

 - Dev

[Cristian Maglie]

Il 31/01/2018 21:53, Develo ha scritto:
> You (Arduino) want to provide specific containers to your user base.
> Let's say a vector of ints and a vector of floats. You could provide this:
>
> using intvector = std::vector<int>;
> using floatvector = std::vector<float>;

....

> Let's say that you don't like the STL object interface, i.e.: method
> names. You could do composition, e.g.: provide something like this:
>
> classintvector
> {
>  private:
>   std::vector<int>mVector;
>  public:
>   voidappend(intval){mVector.push_back(val);}
>   ...
> };

In this case you've already instantiated the template or, in other
words, you are showing an intArray to the user where the templated-type
is already fixed (to be an int). I don't know if you can do something
similar to the use case showed by Thomas Roell without exposing template
syntax to the user.

> About the C++ guru, if you're going to implement a good design, you
> think twice about that design, so that the end users have to think only
> half about its usage.

I just wanted to say that if the solution doesn't come up immediately
probably is not so easy or even possible. Sorry if my previous message
sounded a bit rude, it wasn't my intention.

Anyway I would like to second Paul, if you have some detail to share
before investing much time and effort that would be great.
--
Cristian Maglie <c.ma...@arduino.cc>