Anyone using "data" and "code" subdirectories in sketches?

[Matthijs Kooijman]

Hey folks,

while going over the IDE code, I found some code dealing with a "code"
and "data" subdirectory inside sketches. Apparently, you can add
non-sketch files to a sketch, using Sketch -> Add file, or drag and
drop. Sketch files (.ino, .h, .cpp, etc.) are added to the sketch
normally. If you add a .a, .so, or .o file, it gets copied into the
"code" subdirectory, any one other files get copied into the "data"
subdirectory.

This is clearly a remnant of processing, where I suspect that stuff in
the "code" subdirectory got linked in (the list originally seemed to
include .jar and .class as well). Since the "code" subdirectory isn't
used any more, this part of the code can just be removed, AFAIK.

I think that the support for adding other files to the "data" directory
also doesn't really serve a purpose, and can just be removed too. Adding
a new file, without having any way inside the IDE to see the file added,
or manage it, seems a little pointless. If you need to use a file
manager to view and manage the file, you might as well add it using the
file manager.

*If* some file manager is added to the IDE in a later version, adding
arbitrary files would again make sense, but then this part can just be
re-implemented in a better way, so I see no point in keeping this code
now.

Or is anyone actually *using* this feature, perhaps for a use case I
didn't see?


Gr.

Matthijs

[per1234]

On Tuesday, December 8, 2015 at 11:24:16 PM UTC-8, Matthijs Kooijman wrote:

any one other files get copied into the "data"
subdirectory.

 

I think that the support for adding other files to the "data" directory

also doesn't really serve a purpose, and can just be removed too.

The Library Specification says other files are to be added to the extras folder so if this feature is to be kept I think it should be changed to meet the specification.
 

[Matthijs Kooijman]

Hey,

> The Library Specification
> <https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification>says

> other files are to be added to the extras folder so if this feature is to
> be kept I think it should be changed to meet the specification.

Ah, good point. That specification is about libraries, not sketches, but
it might be good to keep things consistent between them.

Gr.

Matthijs

[Ivan Grokhotkov]

Hi Matthijs,

We do make use of data directory in ESP8266 sketches. Users can put arbitrary files into this directory (html, images, other configuration files), and then use a plugin[1] to upload[2] contents of data directory to the device. Files are stored inside an embedded file system and users can access them directly or make them available through HTTP server.

[1] https://github.com/esp8266/arduino-esp8266fs-plugin

[2] http://esp8266.github.io/Arduino/versions/2.0.0/doc/filesystem.html#uploading-files-to-file-system

I think keeping all related files in a single place (i.e. sketch directory) and being able to upload files through the IDE is a valuable feature.

Regards,

Ivan

[Matthijs Kooijman]

Hey Ivan,

> We do make use of data directory in ESP8266 sketches. Users can put
> arbitrary files into this directory (html, images, other configuration
> files), and then use a plugin[1] to upload[2] contents of data directory to
> the device. Files are stored inside an embedded file system and users can
> access them directly or make them available through HTTP server.

Ah, cool.

However, I do not think you're actually using any of the support in the
IDE though? In your documentation you recommend just creating the data
directory manually and copy in files, without talking about the add
files option, and the plugin just hardcodes the "data" directory (not
using SketchData.getDataFolder().

Given that, I'm still inclined to remove the support in the IDE, AFAICS
that wouldn't impact any of your users anyway?

The only thing you might be using, is that the "data" subdirectory is
now copied when you use "save as", so perhaps keeping support is a good
idea...

Gr.

Matthijs

[Ivan Grokhotkov]

Hi Matthijs,

I think we still rely on the IDE support to copy files along with the sketch.

When we add 'data' directory to an example bundled with the library, we expect that this directory will be copied when users does some edits to the example and clicks "save as".

Perhaps you can remove support for 'data' directory, but then you just need to copy all the sketch subdirectories.

Regarding SketchData.getDataFolder. We were actually using public Sketch.prepareDataFolder method before 1.6.6. In 1.6.6, Sketch.prepareDataFolder became private, and you can not get SketchData instance from Sketch... So I had to resort to some hardcoded paths and reflection to make the plugin compatible with both 1.6.5 and 1.6.6.

Regards,

Ivan

[Matthijs Kooijman]

Hey Ivan,

> I think we still rely on the IDE support to copy files along with the
> sketch.
> When we add 'data' directory to an example bundled with the library, we
> expect that this directory will be copied when users does some edits to the
> example and clicks "save as".

Ah, excellent point.

> Perhaps you can remove support for 'data' directory, but then you just need
> to copy all the sketch subdirectories.

That might be a good idea, though it's a bit more involved.

For now, I'll remove the "code" folder support and leave the "data"
folder unchanged.

Thanks for your input!

> Regarding SketchData.getDataFolder. We were actually using public
> Sketch.prepareDataFolder method before 1.6.6. In 1.6.6,
> Sketch.prepareDataFolder became private, and you can not get SketchData
> instance from Sketch... So I had to resort to some hardcoded paths and
> reflection to make the plugin compatible with both 1.6.5 and 1.6.6.

Right. I'm actually thinking about restructuring this bit, so SketchData will
become a Sketch class without any UI dependencies, and move the GUI bits
of Sketch somewhere else. Not quite sure how this should look exactly,
though.

Gr.

Matthijs

[L. M.]

I looked at the code you pointed out... really creative. I am working on a branch that looks more like a 2.0 than a xx.xx.xx+1. I added some support for editing/viewing .md files as well as images. There is also some basic file management capability. I am curious if there is anything that you'd find useful if you could have your way, I wouldn't mind implementing some of it (e.g. one of the items on my todo is javascript based access to the editor - no need to recompile)?!

thx

( a number of capabilities are listed here )

[Massimo Banzi]

BTW Arduino has selected asciidoc files for Create and not MD files..

that would be part of the new official sketch format

m

--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

--

Massimo Banzi <m.b...@arduino.cc>

[Bill Perry]

I could see having support for a code directory as being very useful for those trying to comply with LGPL copyright requirements when they are wanting to create a closed source product.
They could put the pre-compiled objects of the closed source components into the code directory and then the IDE could be used to rebuild the open source components and link it all together.

Currently, there really isn't a way to comply with the GPL & LGPL licensing requirements when using some closed source components and the IDE.

From those perspectives, having a "code" directory looks very useful.

--- bill

[Bill Perry]

Just as a quick follow up. The reason it is useful is that those using the closed source components could comply with the LGPL license by distributing/posting the sketch directory "blob" which included the "code" directory with the pre-compiled closed source modules.
This satisfies the LGPL requirement of allowing the end user to be able replace/modify/update/rebuild the open source components even when used in/with closed source code.

[Matthijs Kooijman]

Hey Bill,

> I could see having support for a code directory as being very useful for
> those trying to comply with LGPL copyright requirements when they are
> wanting to create a closed source product.
> They could put the pre-compiled objects of the closed source components
> into the code directory and then the IDE could be used to rebuild the open
> source components and link it all together.

Right, that would be useful. However, since the build process does not
currently support this, and so far the Arduino team has explicitely said
not to want this feature (see
https://github.com/arduino/Arduino/issues/4336 and the
issues/discussions it links to). So I do not think keeping the "code"
directory feature intact for this would be really needed (if we ever add
a pre-compiled link feature, it might as well link any objects in the
main scketch directory too).

Gr.

Matthijs

[L. M.]

Is there a custom CSS somewhere for customizing the presentation of adoc files?

lm/