DriveboardApp Software Fork with port to Python 3 and Raster / Fill Improvements Speedups

[Scott (Strataforma.com)]

My understanding is that the main driveboardapp branch is no longer being maintained, since there is a bugfix pull request that has been sitting there since August of last year. Stefan, if you do plan on keeping the main branch updated, let me know and I can make some pull requests to get these changes incorporated upstream.

In the meantime, here is my fork of the driveboardapp repository: https://github.com/scottshambaugh/driveboardapp . I started with FREILab’s fork, since they had already implemented a few of the changes I wanted to make, and built on that. There are still a few things that would be good to add, and some places where things aren’t working perfectly (mostly cross-platform compatibility). I have a list of those at the bottom of this post. My status here is “my setup is working so I may or may not ever get around to making these additional changes”, but pull requests are welcome!

The first order of business was actually getting the original code running as-is. The backend is written in Python 2, and in tracking down dependencies I needed, I found that some of them no longer had Python 2 versions available online. I’m not sure what build environment everyone else had when they got this running, but I wasn’t able to duplicate it. In lieu of trying to recreate missing functions, porting this to a more modern supported codeline looked like it was going to be the easiest way to get things done. Luckily the port to Python 3 was fairly straightforward, though there were some subtle changes to reading and writing serial data from the Arduino that got me stuck for a while. All the functionality is working right now (at least on Windows), and the build environment got a good bit easier – simply clone the repo, install WinPython, then run "pip install -r deploy/requirements.txt".

Compared to the original nortd repository, this fork has quite a few quality-of-life improvements.

• You can now jog in 1 mm increments by holding the ctrl key with the arrows (added by FREILab)
• Your current X-Y coordinates are shown on screen (added by FREILab)
• “Set Offset” now sets the offset to the current head position (added by FREILab)
• The head position is now set in relative dx, dy coordinates (added by FREILab)
• The debug terminal can now be resized, scroll, and will not auto-jump to the bottom of the window if lines print after you scroll up
  
• Functions to print out human-readable versions of the transmitted and received serial commands were added to the source (disabled by default)
• The head will auto-home when the interface is first started (user-configurable)
• Some bugfixes brought in from other repos

What I think the people on here will be most excited about however, are some significant speedups in raster and fill engraving. The default behavior of the machine is to only engrave on the forward pass, which I found infuriatingly slow. So I implemented two new fill modes to the default of "Forward": "Bidirectional" and "NearestNeighbor". Bidirectional simply plucks the low-hanging fruit of engraving rasters/fills on the backwards pass as well as the forwards one. I had worried that backlash might be an issue in running things in reverse, but my edges still look crisp when my belts are well tensioned so this hasn't turned out to be an issue in practice. NearestNeighbor is for fills only, and looks for the nearest engraving endpoint when choosing which line to etch next. 

I have some videos of the two modes up on my personal blog so you can see them in action: https://theshamblog.com/software-mods-to-the-lasersaur/ . NearestNeighbor happens to perform better than Bidirectional for the Lasersaur test image, but whether or not that holds true depends on the shape being engraved. In general I've found dense, complex shapes will be faster with Bidirectional, while simpler, wide shapes will go faster with NearestNeighbor.

Here are the times I measured to fill the default lasersaur test image with a 4000mm/min feedrate and 0.2mm/px resolution:

Forward - 4:01

Bidirectional - 2:33

NearestNeighbor - 2:27 

Known issues:

• Loading up raster image files in Chrome fails (works in Firefox, haven’t tested anything else). This is an issue with the baseline software for me as well.
• Building a Windows executable isn’t currently working.
• Documentation needs to be updated.
• I have not tested this on Mac or Linux.
• The air assist valve remains open in some early-shutdown scenarios even after a power cycle.

Future Improvements:

• Display a numerical time estimate for how long a job will take / how long is remaining in the job.
• Make the “red dot marker” in the web interface snappier.
• Make the fill UI better (button to delete fill, allow changing of pixel size).
• Allow jogging the head away from the magnetic stops (force 1mm steps in case axes are misaligned?)
• Add a UI button to pulse the laser. This will require updating the firmware to implement a “dwell” command, which might be difficult since currently the laser power is tied to stepper movement.
• Calculate and skip over right and left whitespace in raster images, so successive passes don’t go to the edge of the bounding box if not needed.

[Martin Renold]

Nice work. If this was 3 years ago I'd join efforts. Sadly I no longer have so much time at my disposal. I never got around to update my pulseraster fork to support the new (better) web UI.

I'll keep your repo in mind as a better target for PRs in case we do any more of those "obviously good for everyone" bugfixes :-)

Regards

Martin

--

You received this message because you are subscribed to the Google Groups "lasersaur" group.

To unsubscribe from this group and stop receiving emails from it, send an email to lasersaur+...@googlegroups.com.

To post to this group, send email to lase...@googlegroups.com.

Visit this group at https://groups.google.com/group/lasersaur.

To view this discussion on the web visit https://groups.google.com/d/msgid/lasersaur/b2225cec-4b9e-4ddf-8a95-69c2a7a20d0f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Jondale Stratton]

Nice work.  I'll test this out this weekend likely.  Looks like some really nice additions.

To view this discussion on the web visit https://groups.google.com/d/msgid/lasersaur/6118a4a9-1254-4d0a-8c8e-3e166cae7beb%40www.fastmail.com.

[jet townsend]

That's some great work, I'll try and check it out this weekend. I'm
trying to fill a bunch of Etsy orders and need to finish those before I
start tweaking my software.


--
Jet Townsend, IDSA
design <http://www.allartburns.org>
hacking <http://www.flatline.net>
consulting <http://www.functionalprototype.com>

[Johannes F.]

Hey Scott,

this looks awesome! Slow engraving has been bugging us since the beginning as well, but we never found time to actually get into it. I hope we can try this on our machine as soon as possible.

Im also very interested in the human-readable serial commands, since we have been having issues with seemingly arbitrary  "Data" and "Buffer" errors.

Nice to see, that you like our UI changes ;)

Cheers

Johannes

[aaron moore]

Hi

I have lasersaur running driveboard app from a year or so ago.  I would like to try your new version.  I have downloaded it, but cannot figure out how to run it. Are there some instructions for someone not too computer literate?

Mods look great and I can't wait to try it

Cheers

Aaron
 

Your technique is exquisite

Please to check out my new website: www.cnccraft.co.uk
Tel: 01209 890084
Mob: 07805686188
Email:aaron...@linuxmail.org

 

 

Sent: Friday, July 05, 2019 at 5:23 PM
From: "'Johannes F.' via lasersaur" <lase...@googlegroups.com>
To: lasersaur <lase...@googlegroups.com>
Subject: [lasersaur] Re: DriveboardApp Software Fork with port to Python 3 and Raster / Fill Improvements Speedups

--

You received this message because you are subscribed to the Google Groups "lasersaur" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lasersaur+...@googlegroups.com.
To post to this group, send email to lase...@googlegroups.com.
Visit this group at https://groups.google.com/group/lasersaur.

To view this discussion on the web visit https://groups.google.com/d/msgid/lasersaur/aeb7d41b-efe4-4347-b105-acb5c485bea8%40googlegroups.com.

[Scott]

Hey Aaron, here's a step-by-step process for getting this working on Windows:

1. From the github page ( https://github.com/scottshambaugh/driveboardapp ), click the green "clone or download" button, download the zip, and extract it to a folder.
2. Install the latest version of WinPython ( http://winpython.github.io/#releases ) 
   
3. Open up the WinPython command prompt and run python -m ensurepip to make sure pip is installed
4. Navigate to the driveboardapp directory you previously unzipped and run  pip install -r deploy/requirements.txt to get all the python packages you need installed
5. Run python backend/app.py to start the app

To make it quicker to start up, I made a desktop shortcut to run the command "C:\Users\Scott\Documents\WinPython\python-3.7.2.amd64\python.exe" "C:\Users\Scott\Documents\Lasersaur\driveboardapp\backend\app.py" to launch the program quickly. If someone could figure out how to get building .exe's working, that would simplify this process quite a bit.

To view this discussion on the web visit https://groups.google.com/d/msgid/lasersaur/trinity-5e760528-593b-4307-ab15-de997d8dcca8-1563365296976%403c-app-mailcom-lxa04.

[aaron moore]

Hi Scott

Thank you for your reply.

But I am having some problems.... I am running Ubuntu 16.04 (I know. I should be able to do this stuff)

 

When I run 'python driveboardapp/backend/app.py' it says it can't find bottle.

 

when I run 'pip install -r deploy/requirements.txt' I get

 

~/owncloudCNCC/Lasersaurapp/latest driveboard app/driveboardapp-master/deploy$ pip install -r deploy/requirements.txt
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'deploy/requirements.txt'
aaron@holly:~/owncloudCNCC/Lasersaurapp/latest driveboard app/driveboardapp-master/deploy$

 

Can you give some pointers?

Thanks

Aaron

 

Your technique is exquisite

Please to check out my new website: www.cnccraft.co.uk
Tel: 01209 890084
Mob: 07805686188
Email:aaron...@linuxmail.org

 

 

Sent: Friday, July 19, 2019 at 11:11 PM
From: "Scott" <sc...@strataforma.com>
To: lase...@googlegroups.com
Subject: Re: [lasersaur] DriveboardApp Software Fork with port to Python 3 and Raster / Fill Improvements Speedups

To view this discussion on the web visit https://groups.google.com/d/msgid/lasersaur/CAO3CxzuDTqtNSN4SR91zX1P%3DfV4K79TPPV1LR5cih8kjDQ4WkQ%40mail.gmail.com.

[Martin Renold]

I haven't tried it, but it'll be pip3 and python3 instead of pip and python.

Install python3-pip if you don't already have it. Personally I'd also use mkvirtualenv, but that's optional.

Regards

Martin

Compared tothe original nortd repository, this fork has quite a few quality-of-life improvements.

To view this discussion on the web visit https://groups.google.com/d/msgid/lasersaur/trinity-88c4454a-58ca-462e-9e70-7d14d92b6a4d-1563603599260%403c-app-mailcom-lxa10.

[Scott]

Hey Aaron, you probably need to substitute in the commands pip3 and python3. Also it looks like you’re already in the deploy directory so the requirements.txt file should be right there. I haven’t tested these changes in Linux, so I’m afraid I won’t be of much help if you run into things further down the line. 

Cheers -

Scott 

To view this discussion on the web visit https://groups.google.com/d/msgid/lasersaur/trinity-88c4454a-58ca-462e-9e70-7d14d92b6a4d-1563603599260%403c-app-mailcom-lxa10.

[aaron moore]

Hi Scott

Still no joy.  I have installed bottle, but now it says 'No module named dxfgrabber'  Tryed to install python-dxfgrabber but it was unable to locate it.

Any thoughts?

Thanks

Aaron

Your technique is exquisite

Please to check out my new website: www.cnccraft.co.uk
Tel: 01209 890084
Mob: 07805686188
Email:aaron...@linuxmail.org

 

 

Sent: Saturday, July 20, 2019 at 1:01 PM

To view this discussion on the web visit https://groups.google.com/d/msgid/lasersaur/CAO3CxzuYzgZhuz%3Df_KaewGsOdXiEhSXf7Rof60eAE%3DH7GgrB-g%40mail.gmail.com.

[jet townsend]

On 7/22/2019 9:12 AM, aaron moore wrote:
> Hi Scott
> Still no joy.  I have installed bottle, but now it says 'No module named
> dxfgrabber'  Tryed to install python-dxfgrabber but it was unable to
> locate it.
> Any thoughts?
> Thanks
> Aaron
> Your technique is exquisite

it should be in the driveboardapp folder:

$ !find
find ./ -iname "*dxfgrab*"
./.git/logs/refs/heads/dxfgrabber
./.git/logs/refs/remotes/origin/dxfgrabber
./.git/refs/heads/dxfgrabber
./.git/refs/remotes/origin/dxfgrabber
./backend/jobimport/dxfgrabber

[Jondale Stratton]

I spent some time working on this yesterday.  

I have a 1403 machine with the beaglebone black on the driverboard.  

My current working install is some old version of linux that can't even get updates anymore and too old version of python3 so I grabbed a fresh BBB and sd card and decided to start fresh.  I did a fresh install of debian from the beaglebone website, did a clone of your repo, install the requirements.  Pillow didn't install correctly from pip3 so I have that to figure out but that's not a show stopper (just no raster yet).

The stopper is that I can't get the serial connection to work.  I looked at the code which shows I need to change the firmware setting to driveboard1403 in the config file.  I have done that but still nothing.  Looking further at the code it is looking for /dev/ttyO1 which does not exist in my install.  Maybe I need to do something to make that show up.

Everything else seems to work.  I can connect with a browser and see the interface.  Just a No Machine error if i try to do any machine commands.  This probably fixes if i can get serial to work.

I'll play with it some more as I have time .. if anyone has suggestions I'd love to hear it.

--
You received this message because you are subscribed to the Google Groups "lasersaur" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lasersaur+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/lasersaur/728ca229-9791-c70d-5ba8-14da4bafd07e%40allartburns.org.

[Scott (Strataforma.com)]

Hi all, I haven't resolved my issues engraving on my laptop in the other thread yet, but I have have two new fun features on this branch that I was able to test as working on my old computer. As before, you can catch it on github here: https://github.com/scottshambaugh/driveboardapp

The main one is some big speedups to raster engraving. This won't really make a change if you are engraving photos where the entire rectangular area has color data, but if you are engraving stuff with a lot of empty whitespace like blueprints or cut-out images on a white background, this should really speed up your process.

• Any row of pixels that are completely white (ie will not be etched) will be completely skipped rather than be traced over.
• Whitespace on the left and right side of each line of pixels will be skipped (shortening the entire pass on that line).
• "Interior" whitespace that is at least 2x the lead-in/out distance will be moved over at the high-speed seekrate rather than the (likely slower) feedrate. Note that in my tests I haven't noticed that you really need any lead-in or lead-out distance, in which case this will be a big help. But based on your belt tension and specific machine, your mileage may vary.
• "Reverse" is now a direction option in addition to "Forward" and "Bidirectional" for both raster and fill engraving. (Though I don't really see a place where that would be needed.)

Second is some improvements to the serial data printer. Some of the edge cases are handled better now, as is data that is split between two different chunks. And now rather than being a flag in the source code, this is now user-configurable as the "print_serial_data" config option. This won't need to be turned on in the general case but is really useful for debugging during development. And maybe you just want to take a look at the data streaming over the wires.

Additionally, I moved up to python 3.8.0. This seems to have fixed my issues with Chrome not being able to load svg's with raster images embedded in them, which was a nice easy win.

Hope this is helpful for people, let me know if you've got any questions.

Cheers - 

Scott

[soheilhivadnabi]

jondale: did you manage to get it to work with 1403 and beaglebone? I too have problems with serial communication.

[Scott]

Hey Soheil, I got your email and figured I'd respond here just to keep things available for posterity. Unfortunately I'm totally unfamiliar with the older driveboard/beaglebone, so I can't really reproduce issues with those over here on my end. If you can flash with python2 then it might be something simple, if you're able to figure out where in the code things are failing then I can try and see if I can figure why.

One idea - what OS are you using? On Windows I did update the locations of the build/flash executables, you can see those changes here and see if they line up with your Arduino directories:

https://github.com/scottshambaugh/driveboardapp/commit/70eb0d3eb57a7ee5e305bfaa827f91c985b50096

https://github.com/scottshambaugh/driveboardapp/commit/f44c3adc4737b9b6fbc680b610f5d353f62b60cd

Cheers - 

Scott

> Hey, 

> I'm testing out your fork of the driveboardapp, but I'm having trouble with serial communication with the driveboard (v. 14.03 and beaglebone running debian9.5) 

> I'm suspecting it has something to do with python3 / pyserial, as I'm able to flash with python2 on the same serial port.

> Do you have any idea/clue what might be the issue?

> Best, 

> Soheil

To view this discussion on the web visit https://groups.google.com/d/msgid/lasersaur/f317c0a3-ff88-4705-ae3c-7268b813a045n%40googlegroups.com.

[soheilhivadnabi]

I'm running linux distro debian9.5 on a beaglebone black which is connected to the driveboard (14.03). Going through the code, I can't find anything out of place. It should be working, but obviously I'm missing something. Unlike jondale I have the serial port "/dev/ttyO1" in my system (and I've set this up in config.py and config.json) and the port is working as mentioned with the flash.py script (with python2, but which in turn runs avrdude on the serial port). For some reason however the driveboardapp is unable to communicate with the same serial port.

ERROR: Cannot connect serial on port: "/dev/ttyO1"
WARN: Cannot connect to configured serial port.
INFO: Trying to find port.
ERROR: No controller found.

I've tested the driveboardapp with an arduino flashed with the driveboardusb firmware, and it was working just fine, it was even able to automatically find the serial port it was supposed to communicate with.

I also tried connecting an arduino (as an USB-to-serial converter) to the serial header of the 14.03 driveboard, but this was not working.

Well, it's a shame really, mostly because I forked your repository and put in quite a bit of work into revamping the frontend/UI, upgrading it to bootstrap4 standards, and it has turned out pretty sweet, with shiny new buttons and smooth animations.

You can download and test it here if you want:
https://github.com/vanillasoap/driveboardapp/tree/bootstrap4

[Alberto Berti]

>>>>> "soheilhivadnabi" == soheilhivadnabi <soheilh...@gmail.com> writes:

soheilhivadnabi> I'm running linux distro debian9.5 on a beaglebone black which is connected
soheilhivadnabi> to the driveboard (14.03). Going through the code, I can't find anything
soheilhivadnabi> out of place. It should be working, but obviously I'm missing something.
soheilhivadnabi> Unlike jondale I have the serial port "/dev/ttyO1" in my system (and I've
soheilhivadnabi> set this up in config.py and config.json) and the port is working as
soheilhivadnabi> mentioned with the flash.py script (with python2, but which in turn runs
soheilhivadnabi> avrdude on the serial port). For some reason however the driveboardapp is
soheilhivadnabi> unable to communicate with the same serial port.

soheilhivadnabi> ERROR: Cannot connect serial on port: "/dev/ttyO1"

Serial ports on Linux systems do have a "ttyS" prefix... so the first
one is named /dev/ttyS0 and so on /dev/ttyXX are for character terminals
you use when connecting a display and a keyboard to the board.

--
Alberto Berti - Information Technology Consultant
PGP: 9377 A68C C5B5 B534 36BD F20B E3B5 C559 99D6 7CF9

"gutta cavat lapidem"

[soheilhivadnabi]

Thank you for your reply Alberto, I noticed the '/dev/ttySX' ports, and tried setting each one in the config (from ttyS0 to ttyS5), but non worked.

When I'm running a working version of the driveboardapp on the beaglebone (https://github.com/luky83/driveboardapp/tree/debian9.5) it uses the '/dev/ttyO1' port to communicate with the driveboard. Could there be something I need to add/edit in '/boot/uEnv.txt' relating UART?

Currently I've only made these edits:
'disable_uboot_overlay_video=1'
'disable_uboot_overlay_audio=1'
"uboot_overlay_addr4=/lib/firmware/BB-UART1-00A0.dtbo"

I'm using Adafruit BeagleBone-IO-Python library for dealing with GPIO

And btw, I've merged the necessary additions from the luky83 fork (adding support for Adafruit GPIO library) to my bootstrap4 fork.

I might simply need to upgrade from debian9.5 to the latest official img (debian10.3)

[Scott (Strataforma.com)]

I said this earlier: 

>  Note that in my tests I haven't noticed that you really need any lead-in or lead-out distance, in which case this will be a big help.  

but it's come back to bite me.

There is a bug in the firmware such that if you try to execute a raster engraving of length 1 pixel, and you have 0 raster_leadin distance, then the firmware can get stuck. I believe this happens due to the way the raster data is read in inside stepper.c - when there is only 1 pixel then serial_raster_read() does not get called a second time to see the CMD_RASTER_DATA_END block, and when there is no deceleration zone, then consume_data never gets set to true. The combination of both means that the firmware gets stuck in raster_mode, which is blocking, so it never exits out, the rx_buffer fills up, and the machine freezes. This probably went unnoticed until now since no one was trying to raster 1 px images, but the interior whitespace skipping optimizations I put in meant that 1px rastering happens on many images. I don't even want to admit how long figuring this out took me... I was almost ready to admit defeat to the gremlins and give up on the machine.

Unfortunately I cannot seem to fix the issue by modifying the firmware, which means I might not actually understand the root cause. But I do have a workaround - simply set the raster_leadin distance to something greater than 0. 1mm seems rock solid for me.

Scott

[Scott (Strataforma.com)]

Some more updates out now for those interested.

https://github.com/scottshambaugh/driveboardapp 

• Fixed the raster cutting off the end of the image bug
  
• Fixed raster inverting not being implemented correctly
• Put in a raster test file into the default library. The butterfly image is public domain, monarch.jpg from this site.
• Added the ability to pulse the laser! The shiny new red UI button fires the laser at 25% power for 0.1 seconds at its current position, with a 1 second air assist for good measure. I've tested the safety interlocks to ensure that they all still prevent the laser from firing if the door is open, chiller is off, or you hit the 'stop' button (if for some reason you modify the pulse to go longer). Works well, and should be super handy especially if you don't have a red dot!
• Merged in makermusings' pull request to implement saving material presets, which you can apply to passes with the little star button. It's super handy!

Here's the preset library UI, and you can see the preset 'star' and red 'pulse' button back on the right.

Cheers - 

Scott

[Scott (Strataforma.com)]

That screenshot doesn't seem to be showing up, so here it is attached.

Also, I mistyped, the pulse is at 10% power for 0.1 sec, not 25%. I have a 100W laser, so if that's too low to leave a mark for anyone using it let me know and I can update the default or make it configurable.