A small planning for the next IDE releases

[c.ma...@bug.st]

Hi,

I want to share the planned release schedule of Arduino IDE for the next
weeks. I come to
this plan by reading many of the thoughtful inputs here and I hope that
this will help solve (at
least some of) the uncertainty expressed.

About 1.0.x series: there is an IDE 1.0.7 in queue that is, mostly, a
bugfix release for 1.0.6
with the only additional feature of SPI transactions. I hope that this
will be the latest release
for the 1.0.x series.

About 1.5.x series: it's time to go out of beta, everyone is asking for
that, so we'll do it with
the next release starting with 1.6.0 (based on current 1.5.8). No more
1.5.x will be done.

At the end of this process the 1.0.7 will become legacy and the
"official" Arduino IDE will be
1.6.0, resolving the "two IDEs dilemma".

About the IDE GUI/command-line refactoring that I've announced some time ago
(https://github.com/arduino/Arduino/pull/2328) I'm setting up a nightly
build for testing it.
The pull request itself will be merged in one of the next releases
(1.6.1 or 1.6.2).
I would also like to point out that the pull request, even if big, is
just a refactoring:
it doesn't alter the IDE behaviour (unless minor errors) and any 3rd
party core, library
or sketch made for 1.5.x will not be affected.

C

[Suyog Vyawahare]

Any specific date for release?

--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@arduino.cc.

[Paul Stoffregen]

Thanks Cristian. That's excellent news.

I had hoped the large refactoring could be released quickly as 1.5.9
with a relatively short period before declaring it "stable". Even
though it doesn't impact sketches and libraries, it is a pretty huge
change for third parties who dig deeper in the IDE's functionality.
It's also a LOT of big structural change (eg, risk) to release right
after declaring the code "stable".

Is there any chance to accelerate the refactoring and start 1.6.0 based
on this clean slate?

[c.ma...@bug.st]

1.0.7 this week, it's blocked by pull request https://github.com/arduino/Arduino/pull/2381
I'm waiting for some acknowledges before going on. If before Friday I didn't get any response
I'll go on and release 1.0.7.

1.6.0 in two weeks.

take those dates not as a promise but as a rough guess.

For the next releases I don't have a precise timing, consider that the average time for a
release is between 1 and 4 months.

C

To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

--
You received this message because you are subscribed to the Google Groups "Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[Suyog Vyawahare]

Thanks!

[c.ma...@bug.st]

Il 28/10/2014 16:56, Paul Stoffregen ha scritto:
> Is there any chance to accelerate the refactoring and start 1.6.0
> based on this clean slate?

Hey Paul,

I see your point, this may be feasible.
I've pushed a fix on the pullrequest, let me check the build from the
build-bot tomorrow
and I'll decide on that too.

C

[Arnav Gupta]

I have been testing the functions of the IDE extensively with the re factoring. As far as the IDE is concerned, there aren't any regressions I can notice due to the refactoring.

Also the arduino-builder project is nice, I have also noticed a 10% reduction in memory footprint of the IDE after the refactoring. (Could be placebo, but I'd like to hear other's experiences with this too)

If you ask me, making 1.6.0 refactored is a good idea.

Also I have some IDE features coded up that I am feeling a little confused how to make pull requests. Most changes to the IDE will have quite some differences if merged before or after the refactor

Point in example : -
https://github.com/arduino/Arduino/pull/2322

vs
https://github.com/arduino/Arduino/pull/2331

So as long  as the refactor is left hanging, changes to the IDE cannot be merged (because, if you merge them before, then you'll need to rebase the whole refactor branch), and if you refactor before, then you need to modify the patch to suit the refactored IDE.

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Arnav Gupta (championswimmer)        
  Web | Blog | Twitter | Facebook | Github

C

--
You received this message because you are subscribed to the Google Groups "Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@arduino.cc.

[Paul Stoffregen]

On 10/28/2014 01:36 PM, Arnav Gupta wrote:

So as long  as the refactor is left hanging, changes to the IDE cannot be merged (because, if you merge them before, then you'll need to rebase the whole refactor branch), and if you refactor before, then you need to modify the patch to suit the refactored IDE.

My gut feeling is numerous branches tend to discourage open source contributions.

They greatly increase the complexity of simply trying to understand a large open source code base.  They're a minefield of misunderstanding about which branch to use for developing anything new.  Over time, they encourage patch conflicts.  They also tend to create a perception that major contributions become relegated to branches and ultimately neglected.

I know branches do have useful technical purpose.  But perhaps Arduino's current organization, with a total of 17 public branches, might not be the healthiest way to structure an open source project?

[Arnav Gupta]

Yes it does. Even for a seasoned developer who doesn't get confused with 17 branches, I still have 5 changes locally sitting on my laptop (like saving hex, directly flashing a hex, etc) and cannot make pull requests because I am not sure if I should make it on top of 1.5.x branch or on top of the refactored branch :D

As far as Arduino is concerned, I guess except 4 branches, rest all can be (should be?) deleted. They are all more > 1 yr old and were made with temporary purpose in mind, and never deleted.

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  Arnav Gupta (championswimmer)        
  Web | Blog | Twitter | Facebook | Github

--
You received this message because you are subscribed to the Google Groups "Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[c.ma...@bug.st]

Hi,

the following branches on arduino/Arduino repository are outdated and I'll delete them shortly:

- arm-gcc-4.7.4 (obsolete)
- can (deprecated by Collin80's library)
- lib-1.5 (obsolete)
- pre1 (already merged very old)
- update-i18n (already merged)
- master-pull1115 (not working)
- ide-1.5.x-pull1115 (not working)

some other branches are bugfix or pull request tests done by me or ffissore that have been pushed into
arduino/Arduino instead of cmaglie/Arduino or ffissore/Arduino. Even if this is not an technical issue
I understand that this increase the number of different branch displayed on github, so I'll coordinate
with ffissore to move the following branches back into cmaglie/Arduino or ffissore/Arduino repositories:

- ide-1.5.x-rebased-atmel-toolchain -> ffissore
- ide-1.5.x-serial-monitor-max-length -> ffissore
- ide-1.5.x-import-library-from-url -> ffissore
- pin-to-interrupt-macro -> cmaglie
- ide-1.5.x-avr-toolchain-separated-platforms -> ffissore
- scheduler -> cmaglie

This will leave us with the following "active" branches:

- master
- ide-1.5.x
- ide-1.5.x-gui-refactor
- ide-1.5.x-java7

How does it sounds?

C

[Matthijs Kooijman]

Hey Cristian,

> - master
> - ide-1.5.x
> - ide-1.5.x-gui-refactor
> - ide-1.5.x-java7
>
> How does it sounds?

Looks good!

Gr.

Matthijs

[Paul Stoffregen]

On 10/29/2014 03:39 AM, c.ma...@bug.st wrote:

This will leave us with the following "active" branches:

- master
- ide-1.5.x
- ide-1.5.x-gui-refactor
- ide-1.5.x-java7

How does it sounds?

Can "ide-1.5.x" and "ide-1.5.x-gui-refactor" merge to become "master" (defaulting all would-be contributors to where new development should take place), and master become "1.0.7"?

Is it necessary to wait for a release, or could that happen anytime?

[Arnav Gupta]

Those are the 4 branches that we should need/have.

Sounds good to me.

[Matthijs Kooijman]

Hey Christian,

> Is there any chance to accelerate the refactoring and start 1.6.0
> based on this clean slate?

In that same category, there is an open PR to make dependencies between
libraries automatically detected:

https://github.com/arduino/Arduino/pull/2174

I wonder if it makes sense to merge that before 1.6.0, so
libraries/sketches targeting 1.6.x can just assume that this feature is
available? Not sure if it's as big as the refactoring, though (and if we
decide to include both, I suspect this PR needs rebasing on top of the
refactored code).

Gr.

Matthijs

[Paul Stoffregen]

On 10/30/2014 02:16 AM, Matthijs Kooijman wrote:
> Hey Christian,
>
>> Is there any chance to accelerate the refactoring and start 1.6.0
>> based on this clean slate?
> In that same category, there is an open PR to make dependencies between
> libraries automatically detected:
>
> https://github.com/arduino/Arduino/pull/2174

I know it's probably not practical, but this would be awesome to back
port to get into the 1.0.7 release.

Long-term, if sketches with lines like #include <Ethernet.h> are going
to remove #include <SPI.h>, it'd save a lot of end-user pain and
repeated forum questions if it worked on both published versions.

[Johney Five]

I think that obsolete files should be placed in a bin so that when certain people have purpose for them they can still retrieve it... I cant say how many things ive found impossible to find because it was deemed obsolete...though im a newb  to this and there may be a section im not aware of as honestly I am not sure where the things you are all talking about are..

--
You received this message because you are subscribed to the Google Groups "Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@arduino.cc.

[tzi...@gmail.com]

Paul, I would actually advise against this. There are already too many libraries that are written with 1.0.x in mind, and there will be conflict and confusion if this was backported for the last, and deprecating, 1.0.x. Better to keep this drastic change as a "old (1.0) and new (1.6) behavior", otherwise things get much too complex.

I also agree totally, and wanted to +1 the idea of merging all big changes before going 1.6. Not only is it better practice, but it will make it easier for the community to follow through (i.e. For codebender, we won't have to rewrite our system every month for each change), especially considering the new refactoring.

Vasilis

> --
> You received this message because you are subscribed to the Google Groups "Developers" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[c.ma...@bug.st]

Hi,

some small updates on Releases timings:

- 1.0.7 is sill blocked on this pull request https://github.com/arduino/Arduino/pull/2381, it has taken a lot discussion but in the end it seems that an agreement on how to proceed has been reached, so I count to get it merged and released soon.

- for 1.6.0, a bunch of successful tests has been made on the GUI/CLI refactoring (https://github.com/arduino/Arduino/pull/2328) so I'm going to merge it into mainline 1.5.x. After that I'm going to prepare a series of release candidate for IDE 1.6.0, starting from 1.6.0rc1, to quickly iterate over last minute bugs, before releasing the final stable.

C

Il 28/10/2014 17:08, Suyog Vyawahare ha scritto:

Thanks!

On 28 Oct 2014 21:29, "c.ma...@bug.st" <c.ma...@bug.st> wrote:

1.0.7 this week, it's blocked by pull request https://github.com/arduino/Arduino/pull/2381
I'm waiting for some acknowledges before going on. If before Friday I didn't get any response
I'll go on and release 1.0.7.

1.6.0 in two weeks.

take those dates not as a promise but as a rough guess.

For the next releases I don't have a precise timing, consider that the average time for a
release is between 1 and 4 months.

C

[c.ma...@bug.st]

Hi Paul,

Il 29/10/2014 13:00, Paul Stoffregen ha scritto:

Can "ide-1.5.x" and "ide-1.5.x-gui-refactor" merge to become "master" (defaulting all would-be contributors to where new development should take place), and master become "1.0.7"?

Yes, that's exactly the plan.

In detail:
1) a branch called ide-1.0.x is created on the current master
2) ide-1.5.x-gui-refactor is merged into ide-1.5.x
3) master is fast-forwarded to ide-1.5.x

Is it necessary to wait for a release, or could that happen anytime?

There is no technical limit, but we'll do it with the release of 1.6.0.

C

[Michael Shiloh]

It would be great to have a series of tests on fresh installs of operating systems that have caused us issues in the past, namely

windows 8
windows 7

earlier versions of windows?

the recent version of macos that suffered those serial port issues

my fantasy would be that a test computer be given a fresh install of these operating systems, so that the test would have no prior install of drivers etc. do we have such capabilities or should we solicit help from the community?

[Vasilis Georgitzikis]

in codebender, we have a strong computer as a testing rig, where we use Virtualbox and one VM for each version of Windows, from XP all the way up to 8.1, both 32 & 64 bit, as well as all Ubuntu versions (again, 32 & 64 bit) since 12.04 which is LTS.

This has a great upside that you can roll back to a clean snapshot easily, even use snapshots for different versions of driver, or minor versions of the OS (i.e. XP SP3 vs XP SP2).

The downside is that it doesn't help at all with OS X, because it's a pain in the ass to set up in a VM, and it usually entails running cracked versions of the OS instead of vanilla versions that regular folks use.

just my two cents, hoping this will help

[Michael Shiloh]

How well does a VM emulate hardware (USB port) and driver issues? I'd be concerned that I'd be introducing issues that would not be present in a non-virtual system, and missing issues that would be present in a non-virtual system.

M

[Vasilis Georgitzikis]

in general, there's no emulation (afaik). when you "connect" (pair) the USB device to the VM, is shows the correct vid/pid and any message is passes through to the VM.

there IS one issue we had in the past two years. RedBearLab's Blend Micro board, which uses an atmega32u4 board with atmel's signed driver for windows, causes a BSOD in VMs, but works just great in true hardware. of course, you can't check strange issues like USB Hubs getting in the way etc, but what can you do?

Vasilis

[Paul Stoffregen]

On 11/12/2014 01:35 PM, Vasilis Georgitzikis wrote:

in general, there's no emulation (afaik).

This page might be good to review.

https://www.virtualbox.org/manual/ch03.html#idp54070096

In particular, this part about USB might be good to know.

• USB. VirtualBox emulates two USB host controllers, EHCI and OHCI. There is a need for two host controllers because OHCI only handles USB low- and full-speed devices (both USB 1.x and 2.0), while EHCI only handles high-speed devices (USB 2.0 only). The emulated USB controllers do not communicate directly with devices on the host but rather with a virtual USB layer which abstracts the USB protocol and allows the use of remote USB devices.

VMware and Parallels almost certainly do similar USB emulation.

[c.ma...@bug.st]

Il 12/11/2014 19:55, Michael Shiloh ha scritto:
> It would be great to have a series of tests on fresh installs of
> operating systems that have caused us issues in the past, namely
>
> windows 8
> windows 7
> earlier versions of windows?
> the recent version of macos that suffered those serial port issues
>
> my fantasy would be that a test computer be given a fresh install of
> these operating systems, so that the test would have no prior install
> of drivers etc. do we have such capabilities or should we solicit help
> from the community?

Hi Michael,

yes, I'd like to have the community help on that.

Having an automated test machine would be wonderful, but setting up such
system is a real challenge because:

1) A fresh OS should be restored every time you do a test
2) The actions made by the user during the installation are the most
annoying and time consuming to test (say, download, double-click on the
installer, click yes on the dialog box etc.etc.) and should be automated
in some way (how?).
3) Once the IDE is installed, should we consider the test passed? Or
maybe should we check some of the basic functionality of the IDE?
4) Is a "software only" test enough? I guess we should try some basic
hardware test too, say try to upload Blink to the most common board.
5) Using a virtual machine helps a lot in automating the tasks above,
but some tests are not possible or are not as accurate as on a real
computer.

C

[c.ma...@bug.st]

Il 30/10/2014 20:43, Paul Stoffregen ha scritto:
> I know it's probably not practical, but this would be awesome to back
> port to get into the 1.0.7 release.
>
> Long-term, if sketches with lines like #include <Ethernet.h> are going
> to remove #include <SPI.h>, it'd save a lot of end-user pain and
> repeated forum questions if it worked on both published versions.
>

Hi Paul,

this is really not practical, I've currently no plan to backport this
change in 1.0.x.

C

[Rob Tillaart]

Having an automated test machine would be wonderful, but setting up such system is a real challenge because:

1) A fresh OS should be restored every time you do a test

2) The actions made by the user during the installation are the most annoying and time consuming to test (say, download, double-click on the installer, click yes on the dialog box etc.etc.) and should be automated in some way (how?).

3) Once the IDE is installed, should we consider the test passed? Or maybe should we check some of the basic functionality of the IDE?

4) Is a "software only" test enough? I guess we should try some basic hardware test too, say try to upload Blink to the most common board.

5) Using a virtual machine helps a lot in automating the tasks above, but some tests are not possible or are not as accurate as on a real computer.

from my experience:

1) a virtual machine does great jobs here, it takes quite a steep learning curve but it imho is worth the effort (I use VMworkstation on an almost a daily basis). You can have a snapshot of a clean windows 7 machine with some tools (see below) installed.

2) For this you can use AUTOIT - https://www.autoitscript.com/site/autoit/downloads/ -

3) With AutoIT you can also type in "hello world.ino" , select a board and press the compile button.

4) If it passes a software test you are ready for customer BETA (check if it works on different hardware.

5) yes a VM is not a real machine, but what tests are not possible? 

2 cents,

Rob

C

--
You received this message because you are subscribed to the Google Groups "Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@arduino.cc.

[c.ma...@bug.st]

Hi Rob

Il 18/11/2014 16:43, Rob Tillaart ha scritto:
> from my experience:
> 1) a virtual machine does great jobs here, it takes quite a steep
> learning curve but it imho is worth the effort (I use VMworkstation on
> an almost a daily basis). You can have a snapshot of a clean windows 7
> machine with some tools (see below) installed.

A VM is actually much more simpler to *automate* than a real computer.

> 2) For this you can use AUTOIT -
> https://www.autoitscript.com/site/autoit/downloads/ -

Thanks for the link, didn't know that tool, surely worth a look. BTW it
seems to be available only for Windows, are you aware of a solution for
Linux and Mac too?

> 3) With AutoIT you can also type in "hello world.ino" , select a board
> and press the compile button.
>
> 4) If it passes a software test you are ready for customer BETA (check
> if it works on different hardware.
>
> 5) yes a VM is not a real machine, but what tests are not possible?

I'm wondering: how we can check if a driver is successfully installed? I
usually plug in a USB cable with a Leonardo and check if the COM port
appears. This test may be feasible with a VM, but I don't know if
uploading is possible as well, I remember that I had some trouble with a
Windows VM when I tried some months ago.

C

[Rob]

Hello,

For cross-platform automation, Ansible (http://docs.ansible.com/intro.html) is a great choice, and supports windows as well. (Many automation tools don’t support windows or support non-windows.) It uses Python, but that’s generally the extend of the requirements on the “client” machines.

For VM automation, I recommend Vagrant (https://www.vagrantup.com/). It will provision and start a VM, and can bootstrap it with Ansible (or one of many other choices, including direct scripts).

If, however, you are looking at using something like RackSpace or AWS for the VMs, then Ansible will handle the provisioning as well and you don’t need something like Vagrant at all.

Can you tell that this is what I do for my day job? ;-)

-Rob

--
You received this message because you are subscribed to the Google Groups "Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[Bouni]

Hi Rob and Christian,

> Am 18.11.2014 17:01, schrieb c.ma...@bug.st:
> Hi Rob
>
> Il 18/11/2014 16:43, Rob Tillaart ha scritto:
>> from my experience:
>> 1) a virtual machine does great jobs here, it takes quite a steep
>> learning curve but it imho is worth the effort (I use VMworkstation on
>> an almost a daily basis). You can have a snapshot of a clean windows 7
>> machine with some tools (see below) installed.
>
> A VM is actually much more simpler to *automate* than a real computer.
>
>> 2) For this you can use AUTOIT -
>> https://www.autoitscript.com/site/autoit/downloads/ -
>
> Thanks for the link, didn't know that tool, surely worth a look. BTW
> it seems to be available only for Windows, are you aware of a solution
> for Linux and Mac too?

on linux you can use autokey (https://code.google.com/p/autokey/) on
MacOS maybe Automator (http://support.apple.com/en-us/HT2488) or
AppleScript.

>> 3) With AutoIT you can also type in "hello world.ino" , select a board
>> and press the compile button.
>>
>> 4) If it passes a software test you are ready for customer BETA (check
>> if it works on different hardware.
>>
>> 5) yes a VM is not a real machine, but what tests are not possible?

> I'm wondering: how we can check if a driver is successfully installed?
> I usually plug in a USB cable with a Leonardo and check if the COM
> port appears. This test may be feasible with a VM, but I don't know if
> uploading is possible as well, I remember that I had some trouble with
> a Windows VM when I tried some months ago.
>
> C

I think it should be possible to test every thing like this:

You have 3 virtual machine images (win, linux, mac), the host has a
Arduino board connected via USB, and the USB is bridged to the Guest
system.
When the Guest system starts up it executes a script
(AutoIt,autokey,Automator) that checks if the Virtual Serial port is
available.
Then it executes the necessary tasks to make sure the new build works
fine.
Maybe you can store the result on a share which is located on the Host
system, so you can check the outcome automatically.

Bouni

[Michael Shiloh]

Comments inline

On Tue, Nov 18, 2014 at 7:15 AM, c.ma...@bug.st <c.ma...@bug.st> wrote:

Il 12/11/2014 19:55, Michael Shiloh ha scritto:

It would be great to have a series of tests on fresh installs of operating systems that have caused us issues in the past, namely

windows 8
windows 7
earlier versions of windows?
the recent version of macos that suffered those serial port issues

my fantasy would be that a test computer be given a fresh install of these operating systems, so that the test would have no prior install of drivers etc. do we have such capabilities or should we solicit help from the community?

Hi Michael,

yes, I'd like to have the community help on that.

Having an automated test machine would be wonderful, but setting up such system is a real challenge

Yes, but for those of us who teach users on Windows and have to deal with all the issues that come up, having confidence in the process and knowing the failure modes would be worth the challenge. I suspect that the same is true for those of us who answer support calls.

 

because:

1) A fresh OS should be restored every time you do a test

Yes, absolutely. I'm assuming that people who volunteer to help will have a spare computer on which they will install a fresh OS for each test.
 

2) The actions made by the user during the installation are the most annoying and time consuming to test (say, download, double-click on the installer, click yes on the dialog box etc.etc.) and should be automated in some way (how?).

The procedure to follow must be whatever written instruction we provide on our "Getting Started" page. This way we test the release as well as our instructions.
 

3) Once the IDE is installed, should we consider the test passed?

Absolutely NOT

 

Or maybe should we check some of the basic functionality of the IDE?

Upload blink example, observe blinking, change delay() rate amount, upload again, observe change in blink rate.

From my experience in workshops and classes, once users get past that, everything else works.

4) Is a "software only" test enough? I guess we should try some basic hardware test too, say try to upload Blink to the most common board.
5) Using a virtual machine helps a lot in automating the tasks above, but some tests are not possible or are not as accurate as on a real computer.

C

I will draft such a request and see what sort of response I get from the community. It is only meaningful if we get some decent response.

I have some questions:

1. Other than Windows 7 and 8, what other versions of Windows do we support? XP? Vista?

2. Tom Igoe, has the issue you were having with your new Mac been resolved?

3. Should we test Macs too or just Windows?

 

--
You received this message because you are subscribed to the Google Groups "Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to developers+unsubscribe@arduino.cc.

[Tom Igoe]

> 2. Tom Igoe, has the issue you were having with your new Mac been resolved?

Yes, it had to do with the serial change that came about 1.5.5 to 1.5.6, if I recall. 1.5.8 works solidly now.

t.

[David Cuartielles]

> --
> You received this message because you are subscribed to the Google Groups "Developers" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

[Tom Igoe]

Doubt it. I have never seen the white screen of death, what does it look like?

t.

[Ricardo JL Rufino]

A support Auto-Complete would be welcome?

Is there anything being done to it?

I'm doing some implementations and is starting to get cool, was initially based on keywords.txt, but you can not know the methods of each class, I'm finishing to implement a parser that would read files.he generate the list of methods for each class

[c.ma...@bug.st]

Hi Ricardo!

auto-complete is a feature that has been requested many times, but we didn't had the time/resources to actually implement it. May you give some more details on how you did it, or maybe a link to a demo?

I've merged yesterday a big refactoring of the IDE GUI code, and now I'm considering the adoption of RSyntaxTextArea widget as suggested here:
https://github.com/arduino/Arduino/issues/1590#issuecomment-37107055

By the way, this widget provides only on the GUI, the "C++ parser" should be implemented in some other way and IMHO it's the most difficult part.

C

[Brian Cook]

> By the way, this widget provides only on the GUI, the "C++ parser"
> should be implemented in some other way and IMHO it's the most
> difficult part.

The Embarcadero (previously Borland) folks use the actual compiler to
parse for code-completion and source code coloring.

Could you do something similar with GCC? Maybe something involving
-fdump-tree-all / -fdump-rtl-all?

- Brian

[Matthijs Kooijman]

Hi folks,

> Could you do something similar with GCC? Maybe something involving
> -fdump-tree-all / -fdump-rtl-all?

I think that clang is actually often used for this, I think it exposes a
clean interface to the parsed code. Not sure how well this works from
java, though. Also, llvm doesn't actually support AVR, not sure if clang
could be made to work (even though it doesn't need code generation, it
does need to know stuff like pointer and integer sizes...).

Gr.

Matthijs

[Ladislas de Toldi]

Matthijs is right, clang is often used for code completion and syntax checking in place of gcc. The API is said to be far more cleaner and easier to play with.

A good example of this is YouCompleteMe, a code completion plugin for Vim. I’ve been using it for more than a year now and just fell in love with it. I use it a lot for my Arduino projects. It needs to be tuned but once it’s done you just can’t live without it. Here is my current configuration. Basically the python script looks for my libs to be included with -I/path/to/my/libs

It relies on ycmd a code completion and comprehension server. That’s the part doing all the magic and that could be included inside the Arduino IDE to provide those features.

Another project relying on clang is SublimeClang, a C/C++ code completion and syntax checking plugin for SublimeText. Before switching to Vim, I’ve used it for two years, on every kind of projects including Arduino. The configuration is much more easier with a simple .json file. Here is an example

Those two projects are good examples of what could be done for the IDE, I hope it helps.

​

[tzi...@gmail.com]

We're also using clang on codebender and, while it took us a long time to get it working right and fix all strange behavior, it's fairly straightforward after it's done.

I don't have any free time for coding these days, so I'm afraid I won't be able to help port this to the IDE, but anyone interested can take a look at https://github.com/codebendercc/compiler and port it to arduino. I'm happy to help with questions too, I just don't have free time to work on t myself.

Vasilis

--

[Ricardo JL Rufino]

I am finalizing the implementation of Auto-Complete and some improvements in code editing, I finish up Saturday and do a fork / pullrequest.

I'm using Eclipse CDT Parser, which is used in the IDE, do the analysis of libraries (.h files) and create an index.

it's already pretty cool, but lacks some adjustments.

Detail: it is pure java!

Ricardo JL Rufino

----------------------------------------------------------------------
CriativaSoft - Desenvolvimento de Sistemas com Qualidade
Analista e Engenheiro de Software

Skype: ricardojlrufino

Linkedin: br.linkedin.com/pub/ricardo-rufino/5a/696/81a

[Ladislas de Toldi]

@Ricardo: is it a little like ctag?

[Ricardo JL Rufino]

I did not know Ctags, what I do is make the parser to generate a more simplified version of Eclipse ASTTree, and I think in serialize to a file within each library folder.

If the library already has this aquivo (lib.index) use it, but will be generated when importing the library (in the case of user libraries.)

And in the main Sketch, I'm testing some ideas for no making a full parser on every modification.

For example when the User declare a variable, it should appear in the autocomplete ... I think this is the most complicated, but let's do it ... =]

2014-11-20 12:28 GMT-03:00 Ladislas de Toldi <ladi...@leka-smarttoys.com>:

@Ricardo: is it a little like ctag?

--

[Minhaz A V]

Hi, I started to work on Arduino today and realised auto completion is missing. Is this work still going on? or it has been implemented in the beta version.
I'm using version 1.0.6

[Ricardo JL Rufino]

The development has been completed, now believe that the team is evaluating the integration.

If you want to help test the src is the link:

https://github.com/CriativaSoft/Arduino/tree/ide-1.5.x-autocomplete

[Minhaz A V]

little heads up on how to build this?

----------------------------------------------------------------------------

Kind Regards,

Minhaz | My blog

[Paolo Paolucci]

--> https://code.google.com/p/arduino/wiki/BuildingArduino

[c.ma...@bug.st]

Il 17/12/2014 19:56, Ricardo JL Rufino ha scritto:
> The development has been completed, now believe that the team is
> evaluating the integration.
>
> If you want to help test the src is the link:
>
> https://github.com/CriativaSoft/Arduino/tree/ide-1.5.x-autocomplete

Hi Ricardo!

since the following review may sounds somewhat negative, I want to make
it clear that it's not: you did an amazing job on a very difficult task
and nobody, until now, has gone so close to the finalization of the
auto-complete feature.

My apologies for the delay, your commit is huge and it took a while to
review (and the Christmas holidays didn't helped).

In general, we prefer to have a branch with many commits, each one
operating a single small change, this helps a lot the understanding,
instead of having one huge commit that changes everything.


Functional tests
----------------
Overall the patched IDE seems to work fine within basic usage (open
IDE/write sketch/upload) but a number of intermittent failures appears
if I start tinkering, in particular I've got some NullPointerExceptions
when trying to verify/upload a sketch with multiple tabs opened or with
many sketches opened at the same time.

Also, something strange is happening when I change board: if I switch
from Arduino Uno to Arduino Due for example, the autocompletion list is
not updated (instead the autocompletion list for Arduino Uno is showed).
If I restart the IDE, leaving the Due selected, then the completion list
is updated.

I've seen also a number of small bugs, for example:
- the auto format doesn't keep the cursor in place
- if you swap two lines with Alt+Up/Down the Undo is not correct

This is what I discovered in a couple of hours of testing, I guess that
there is more to discover and fix and iron out.


Building the IDE
----------------
The ant build.xml is not working and we need it to create the
distribution files for mac win and linux.
On your branch the ant build fails because some java 1.7+ only
constructs has been used. BTW, after fixing the trivial syntax errors,
the IDE still doesn't start. I managed to run it from eclipse by
tweaking the run settings in some way, but this needs some more
investigation.


Code design
-----------
You started working on a fork made before the merge of the GUI
refactoring in https://github.com/arduino/Arduino/pull/2328. This may be
a big problem.
I've tried to rebase your commit on top of the latest ide-1.5.x and it
shows some issues, in particular the autocompletion seems to store data
(and to add data processing methods) into GUI widgets. This goes against
what we tried to do in #2328, i.e. split GUI code from
Builder/Uploader/Logic code.

You also provided a jar with the C++ source code analyzer
(cplus-libparser.0.0.1.jar) that is used in the Arduino IDE: this is
great but there is still a lot of autocompletion logic inside the IDE.
What I'd expect is that the IDE "calls" the autocompletion-plugin when
needed and the plugin returns the list of autocompletions that are just
"showed" to the user.
We should probably define an interface API that the autocompletion
plugins must implement in order to be "plugged" in the IDE. This will
help to keep the IDE GUI clean from any autocompletion logic. I like
this approach more, even if more data-passing is required between IDE
and autocompletion plugin.


Where to go from here?
----------------------
The patch cannot be merged as is and it needs some more work, at the
very minimum it must be merged with the latest ide-1.5.x branch and make
the ant build working.

A plan to accelerate the adoption into 1.6 may be:

1. start from a clean and updated ide-1.5.x branch: after #2328 the IDE
is splitted into two projects (arduino-core and app) ant it's much
easier to keep the logic (arduino-core) separated from the UI (app).

2. pull your implementation of RSyntaxTextArea without all the
autocomplete stuff into the UI. This, hopefully, can be reviewed and
merged quickly.

3. define an IDE plugin-API interface that is needed to make
autocompletion work (now that you experimented with autocompletion it
should be easier for you to define such API)

4. change your current autocomplete plugin to implement the above API,
in a separate project to avoid interference between the GUI and the plugin.

Said that, I'd like to know what do you think, if that makes sense, if
you need help in something, etc.

And, again, thanks for your hard work on this.

C


PS: F.Fissore just sent me a patch to fix the ant build.xml, it looks
like there were missing some gif icons (I didn't tried it yet BTW), it's
in attachment.