Arduino IDE 2.0

126 views
Skip to first unread message

Ken Perry

unread,
Oct 30, 2021, 1:41:35 AM10/30/21
to devel...@arduino.cc

 

I am pretty disappointed.  I just finally had time to try the latest Arduino 2.0 IDE Beta.    The accessibility and short cut keys was bad in Arduino IDE 1.x.  It is worse and unusable in the latest builds of beta 2.0.    There seems to be some attempts to glue accessibility on but I can’t even use the menus with either of the most popular screen readers. 

 

Further more the forums have no posts about shortcuts worth commenting on and nothing I can find on accessibility all though it looks like someone started adding accessibility.

I am more than willing to help but so far it is unclear even where to get started with this web IDE>  Web IDE’s are awful to start with for blind users without adding a lot of shortcuts to jump from one dialog to the other.  Normally the menus work though and in this case that is not true. 

 

Anyway if anyone has input to what is planned here and why there is not more already working I would love to hear it.

 

I will point out being accessible is how this stuff gets used in schools.  I had started working with some Teachers of the Visually impaired on these environments because they are talking about using it in their school system.  If the IDE is not accessible though the school systems will be forced to go to other things by the laws in place.

 

 

 

 

Alessandro Ranellucci

unread,
Oct 30, 2021, 5:55:09 AM10/30/21
to devel...@arduino.cc
Hi Ken,

that’s a very important topic indeed and I’m glad you raised it. The IDE 2.0 is a huge project, and lots or work still needs to be done. On your question about why there’s not more already working in terms of accessibility, it’s just because the team has been busy implementing other core features like serial plotter and pluggable discovery and tens of other things that are needed as well.
But it’s a collaborative effort, so contributions and feedback from the community are very precious to increase the development velocity and improve the overall maturity of this new IDE.

Could you share your observations? What dialogs are not easily reachable? What shortcuts do you think are missing?

Thank you! :)

 - Alessandro


--
You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.
To view this discussion on the web visit https://groups.google.com/a/arduino.cc/d/msgid/developers/005e01d7cd50%24cc0d2410%2464276c30%24%40blinksoft.com.

Charles Papir

unread,
Oct 30, 2021, 8:44:30 AM10/30/21
to devel...@arduino.cc, Alessandro Ranellucci

There are several things that I found problematic on the IDE 1.n

1. Dialog output Link MAP "Yes", "No".Add a feature to create a linker MAP of entry points, length, address etc.

2. Dialog to control ".bin" and ".hex" files (produce ".bin" or ".hex" or both, with bootloader or no-bootloader or both). To produce a ".bin" or "hex" file to OTA. Sketch->Export compiled Binary did NOT create the "***.ino.with_bootloader.arduino_zero" all the time a more sturdy process to generate ".bin" is needed

3. Issues in that the compile sketches do not correctly use all the program memory available. If you can add a way of changing these type of parameters it would be great.

4. Simple drop down to make several compile options such as define "#define name.."

Ken Perry

unread,
Oct 30, 2021, 3:16:15 PM10/30/21
to devel...@arduino.cc

First I am a Senior Software Engineer on projects like the BrailleBlaster  (Brailleblaster.org) and much more.  So I am not without the ability to help.     

 

First the system to make reports is unclear and might be accessible but it is not very usable for those of us with Screen readers.      I tried to search for any issues about accessibility and so far have been able to find nothing.  I even tried to find issues on shortcut keys and the only thing I was able to find was something to do with deleting and moving lines of code.  One thing that might be nice is if there was some more clear ways of getting involved in the project.  I would not mind being part of fixing the problem.

 

Second you asked what I have seen as the problems.

 

  1. No way to get to the error window that I can find. 
  2. Open the menus and the menus don’t focus.  Note this is with NVDA or Jaws the two most popular screen readers.
  3. The only thing I can do right now is open code and edit it.  I might be able to compile it and uploaded it with the short cuts but so far I have no idea because I can’t get to the error / messages window to tell me what is going on.
  4. There is no way to get to any other window but the main window.
  5. I can change from tab in the edit box and out but when I switch with control +m and then try to tab around the interface nothing happens.

 

That is just a short list.  I am sure there are more once I am able to move around a bit in the IDE>   Do you guys have anyone with accessibility knowledge working on this project?  I can tell you if you try to plug accessibility in last it is only going to make things worse.  If you make views and stuff that have no accessibility built in to start then it is going to make it harder to make those links later.   Heck something like alt0 -alt 9 would help if each one focused different windows.  It might not be the best way to go about things but it would be a short cut start.  My question is though why the heck are the menus so screwed up.  They should work if nothing else does.

 

 

  1. There is no view window to open and close windows that I need or do not need.  Not that I can get to them anyway.

Ken Perry

unread,
Oct 30, 2021, 3:38:36 PM10/30/21
to devel...@arduino.cc

 

I had to reply one more time after I read your email again.  You said this:

 

“that’s a very important topic indeed and I’m glad you raised it. The IDE 2.0 is a huge project, and lots or work still needs to be done. On your question about why there’s not more already working in terms of accessibility, it’s just because the team has been busy implementing other core features like serial plotter”

 

 

My answer is this.  If you were working on this as you are and lets say the screen was blurry, upside down and sideways.  Would there be more important features?  If the IDE is not accessible there is no important features.  The interface should work for those it is intended for first before the “”more important features” work.   What if it turns out that the accessibility can’t be built in later.  What will have been important then when the schools will not use it because of government mandates for into be made accessible.  I understand this is not just for schools but it is a major portion of people using these kind of tools.

 

Anyway I just checked on github for current tickets and it looks like there are a few that go back quite a ways.  Here they are:

360        OPEN     Accessibility: Screen reader semantics and keyboard navigation could use some TLC            topic: code, type: imperfection, topic: accessibility            2021-09-22 16:07:48 +0000 UTC

166        OPEN     Use of disappearing popups (toasts) for upload/compile errors is inaccessible to screen readers     topic: code, type: imperfection, topic: accessibility               2021-09-22 15:48:38 +0000 UTC

165        OPEN     Keyboard navigation of toolbar is out of order     topic: code, type: imperfection, topic: accessibility               2021-09-22 15:48:13 +0000 UTC

153        OPEN     add accelerator/ shortcut keys for IDE main menu on Windows   type: enhancement, topic: code, topic: accessibility      2021-10-29 04:53:05 +0000 UTC

 

That are just some I find by using the command line gh client for github.  I will be seeing if there is something I can do but I am not sure electron is that accessible to start with.  Microsoft is using it in their make code project and even the Microbit development environment has problems even though they are farther along than the Arduino IEDE.  I can at least use the menus, pop up the tool  box and switch between python, java script and blockily on that IDE so maybe electron interface can be saved.

 

Ken

 

From: Alessandro Ranellucci <a.rane...@arduino.cc>
Sent: Saturday, October 30, 2021 5:55 AM
To: devel...@arduino.cc
Subject: Re: [Developers] Arduino IDE 2.0

 

Hi Ken,

Ken Perry

unread,
Oct 30, 2021, 4:05:09 PM10/30/21
to devel...@arduino.cc

Ok I went in to see how to build this.  I got this out of the build.md

If you€™re familiar with TypeScript, the [Theia IDE](https://theia-ide.org/), and if you want to contribute to the

project, you should be able to build the Arduino IDE locally. Please refer to the [Theia IDE prerequisites](https://github.com/theia-ide/theia/blob/master/doc/) documentation for the setup instructions.

 

First I am not familiar with it but when I go to the page it references there is no requirements and no real clear thing to understand what I need to do to setup the build.  I have the one git repo but it says there are two others which I do not yet have.  So I think the build instructions are a bit lacking in this repo.

From: Alessandro Ranellucci <a.rane...@arduino.cc>
Sent: Saturday, October 30, 2021 5:55 AM
To: devel...@arduino.cc
Subject: Re: [Developers] Arduino IDE 2.0

 

Hi Ken,

Alessandro Ranellucci

unread,
Nov 1, 2021, 3:06:41 PM11/1/21
to devel...@arduino.cc
Hi Charles,

can you provide more context about your points? It would be helpful to understand the use cases (i.e. what’s your goal) and exact steps to reproduce the issues (settings, platform, hardware...).

Also, it would be more appropriate to post these proposals in the relevant GitHub repositories, otherwise they will likely get lost in this mailing list:

Thank you!

 - Alessandro



Paul Stoffregen

unread,
Nov 1, 2021, 4:02:42 PM11/1/21
to devel...@arduino.cc
How are those 4 things about advanced compiler & linker options related to accessibility?

Alessandro Ranellucci

unread,
Nov 1, 2021, 4:03:32 PM11/1/21
to devel...@arduino.cc
Hi Ken,

thank you for sharing that report. To make sure your points are properly tracked I copied them to an issue in GitHub:

There are already a bunch of accessibility-related issues that we group with a dedicated “accessibility” label:

Others are also tracked in the upstream Theia repository:

Your report about build instructions not being clear is extremely relevant and we’ll make sure they are improved, since it’s very important to enable anyone to contribute to development.

Just one question: what is your operating system?

- Alessandro

Alessandro Ranellucci

unread,
Nov 1, 2021, 4:16:14 PM11/1/21
to devel...@arduino.cc
One more point about accessibility: in order to provide a workaround while the IDE is being worked on, we have been working hard to expose all the features of the IDE from command line: you can code with your editor of choice, and then call arduino-cli from the terminal to compile, upload, install libraries and so on.

Many users are not familiar with command line tools, of course, but that's still an option that wasn’t available before, so one step forward. Of course, any feedback about the command line interface is welcome too. :)

 - Alessandro





Ken Perry

unread,
Nov 1, 2021, 5:01:35 PM11/1/21
to devel...@arduino.cc

My operating system is a hard question.  It is what I need it to be.  I build on Mac, Linux, and Windows.  So I figure one of them should be rather easy to setup for a build like Arduino IDE>     I am mainly a c++, C, Asm, Java, python programmer but back with Ginger bread Android I had to write a JavaScript screen reader for braille Android phone.  I can get back up to speed on JS pretty quickly but it looks like the world has changed past just node.js which I have wrote a few things for.    I feel like I am a little lost when I read something like go read a document on something totally different to learn how to build.  I would think there should be a simple list of steps to get things in a build ready state.   Currently I do about 60% of my coding in Windows but I split the remaining time in Linux and Mac.  Not to mention the work I do in Windows is split between GUI, cmd, and Ubuntu wsl2.  Again I do want to do more than just wine so if someone can help with getting me going I will see what I can do. 

 

Ken

Ken Perry

unread,
Nov 1, 2021, 5:02:26 PM11/1/21
to devel...@arduino.cc

I think his list is not very related to what I started this on.  It sounds more like general problems.

 

From: devel...@arduino.cc <devel...@arduino.cc> On Behalf Of Paul Stoffregen
Sent: Monday, November 1, 2021 4:03 PM
To: devel...@arduino.cc
Subject: Re: [Developers] Arduino IDE 2.0

 

How are those 4 things about advanced compiler & linker options related to accessibility?

Ken Perry

unread,
Nov 1, 2021, 5:16:10 PM11/1/21
to devel...@arduino.cc

I am fine with and love the Arduino-cli. 

 

Here is the problem. I work in a company which makes things for the blind. We do Services, software, and devices.  We make the things the schools use.  We have recently added accessibility to an Arduino kit and are creating a more accessible Arduino kit that has less blinking lights and more sounds, vibrations, motors and servors while teaching the same things.  Even without those kits the Teachers in schools don’t want to learn something different when working with blind students even though they sometimes have to.  With that said we try to direct them to what everyone uses.  So far there is nothing that is easy for a teacher to use and make it  so blind kids can work with their sighted counter parts without dropping them to the command line. 

 

Note I am a software engineer and love to live in command lines of all kinds but in a school environment teachers want the quick and easy way to get kids started.  That is why things like VSCode would be great but while I have installed the extension for Arduino and the cli.  So far I have not been able to set the variables I need to without assistance to make it a workable IDE.  I use visual studio with no problem but these web based apps are awful with blind access software.  I hope they continue to get better but so far web based applications are not great. 

 

WE had one of our coders add a couple things to make the old Arduino IDE  library and board manager more accessible it went in in version 1.8.10.    Unfortunately the Arduino IDE uses non native java GUI which is yet another thing screen readers have trouble with.  It takes a special access bridge to make it work with screen readers and that takes extra setup.  With that said once they get the access bridge added the old Arduino ID beyond 1.8.10 works rather well with screen readers.  The problem is schools are shifting to VSCode, online make code, and amazingly they are looking at the new Arduino 2.0 beta already.  All of which have serious accessibility problems.    I think the thing they like most about the new 2.0 is how it helps set things up if you ad a board, library, etc.…

 

I guess I have  rambled all that to say this.  Sighted teachers want to use simple IDE’s to teach all the kids and they don’t have time to learn different methods like a command line in beginning computer classes unfortunately.  We definitely get kids into Arduino-cli as soon as they don’t need guidance but sometimes that is not as fast as we might want with as quick passed schools are now days.

 

Anyway long way to say.  Love Arduino-cli but it doesn’t fit what we need it for.

 

 

From: Alessandro Ranellucci <a.rane...@arduino.cc>
Sent: Monday, November 1, 2021 4:16 PM
To: devel...@arduino.cc
Subject: RE: [Developers] Arduino IDE 2.0

 

One more point about accessibility: in order to provide a workaround while the IDE is being worked on, we have been working hard to expose all the features of the IDE from command line: you can code with your editor of choice, and then call arduino-cli from the terminal to compile, upload, install libraries and so on.

 

Many users are not familiar with command line tools, of course, but that's still an option that wasn’t available before, so one step forward. Of course, any feedback about the command line interface is welcome too. :)

 

 - Alessandro

 

 

 

 

 

--

You received this message because you are subscribed to the Google Groups "Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to developers+...@arduino.cc.

Ian Katz

unread,
Nov 2, 2021, 9:26:42 AM11/2/21
to devel...@arduino.cc
It occurs to me that this thread is describing what is essentially a
regression in one particular feature (accessibility), and that
automated tests of various flows through the IDE might have made this
regression more evident to sighted developers.

Ken, do you know of a screen reader that can be integrated into unit
tests and/or Continuous Integration?
> To view this discussion on the web visit https://groups.google.com/a/arduino.cc/d/msgid/developers/00a801d7cf65%24af655820%240e300860%24%40blinksoft.com.

Ken Perry

unread,
Nov 2, 2021, 10:15:11 AM11/2/21
to devel...@arduino.cc
Yes and no. There are free screen readers such as the one that comes with Windows like Narrator, Then the more popular one which is NVDA. Then for Mac Voiceover. For Linux there is Orca. The problem is I am not sure you can automate using them. With that said there are some checks that can be done with automation without having a screen reader. A lot of it is focus testing and keyboard navigation just to get started with. I will talk to our QA's and see what they suggest but as for me any tests that can be done to test keyboard focus will help with accessibility for more than just blind. Anything that requires a mouse can seriously mess with a lot of accessibility. Motor skills and blindness are just two. It is important to focus a window of choice. I understand that a sighted person can see the serial monitor, the code window, and the error window all without changing the actual focus. A blind person can not. We can only deal with one window at a time so the focus actually needs to change. As for a person with motor skill issues. It is hard to use the mouse and focus a window when your hand can not control the mouse which means using the keyboard is a better method to switch focus from one window to another.


So if you can't do tasks by testing using keyboard commands then there is an accessibility issue. I think you can automate that pretty easily For example a test might be:

Open file
Input some code.
Build code
Check errors.
Push to board
Open serial monitor
Focus view of serial monitor for arrow navigation.
Check line 5 of serial output using key navigation.

If you can't do that all by keyboard then you have a serious issue.

Also it is important to understand if you have windows that you can interact with but can't type in it still needs to be able to get keyboard focus for access software of all kinds. For example if you have an error window that you can't type in but you can click on an error to have you take the you to an error in the code. Multiple things need to happen.

First you need to be able to focus that window with the keyboard
Second you need to be able to arrow to each error.
Third you need to be able to hit enter on it rather than double click. That or be able to use something like the application key rather than right click.
Finally once you select it with enter it should change your focus to the line of text in the editor not just select the text.

If you want to see how bad this really is with windows run narrator with ctrl-windows-enter and you can turn it off with the same key press. Then try to do some tasks. Even simple tasks using the keyboard. Try to go to the menu bar which should activate by pressing either alt or in some of these web UI you have to use F10.


If you are using Mac it might be better I will have to go try it with mac but if you want to try it with Mac Press cmd-f5 to turn on voice over and cmd-F5 to turn it off. Again once your in just try to do some stuff with the keyboard.


If you are using Linux you will need to install Orca before you can test with it but if you’re a Linux user I am sure you can figure out the steps or search them out.


Now after saying all that mess. Here is something that might even help more than my rambling:

Accessibility testing toolkit for desktop applications | by Amoli | UX Collective
https://uxdesign.cc/accessibility-testing-toolkit-for-software-applications-77f1a65d694b


-----Original Message-----
From: devel...@arduino.cc <devel...@arduino.cc> On Behalf Of Ian Katz
Sent: Tuesday, November 2, 2021 9:26 AM
To: devel...@arduino.cc
Subject: Re: [Developers] Arduino IDE 2.0

To view this discussion on the web visit https://groups.google.com/a/arduino.cc/d/msgid/developers/CAKLOAe17VaKjDLM3yBV2%3DCpM4RsNnwGsFgKtXfY%2Bstkh-rh7ew%40mail.gmail.com.

Reply all
Reply to author
Forward
0 new messages