Building on armv8.

[Daren Schwenke]

Tried the packaged version (the newer one).  Provided I set a memory limit, works, except for when I try to add the openpnpcapture camera.

So I pulled that down and built that locally.  Well, it outputs a .so file, so I put that a few logical places with no luck.

So.. I pulled down openpnp itself, and built with maven.

Builds, works, again, except for complaining about native libraries still.

Unable to load library 'openpnp-capture': Native library (linux-arm/libopenpnp-capture.so) not found in resource path.

I take it it's just because we have bundled in native libraries, and the builds don't do arm.

So where do I put it?

[Jason von Nieda]

You should be able to put it in your system PATH under linux-arm/libopenpnp-capture.so, so for instance, maybe somewhere like /usr/bin/linux-arm/libopenpnp-capture.so. I haven't personally tried this, but that's how it should work, I think.

See the docs for https://java-native-access.github.io/jna/4.2.1/com/sun/jna/Native.html for more details. That is the class that is used to load the libraries.

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/a7311e8e-6cf7-4422-89e1-731237d3e844%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Daren Schwenke]

Thank you Jason.  I've stopped for today, but I will give this a shot in the morning.

[Bernd Walter]

On Sat, Aug 18, 2018 at 08:26:02PM -0700, Daren Schwenke wrote:
> Thank you Jason. I've stopped for today, but I will give this a shot in
> the morning.

This is what I did to get it running on FreeBSD:
I've installed packages:
opencv-2.4.13.1_6 Open Source Computer Vision library
opencv-core-2.4.13.1_1 Open Source Computer Vision library
opencv-java-2.4.13.1_1 Open Source Computer Vision library

And modified the opencv in pom.xml:
<dependency>
<groupId>local</groupId>
<artifactId>opencv</artifactId>
<version>2.4.13</version>
</dependency>

I also removed all nu.pattern.OpenCV.loadShared(); calls.

I've attached my full local diff - some are unrelated, but the commits
are spread out and uneasy to get them all at once.

> >> email to openpnp+u...@googlegroups.com <javascript:>.

> >> To post to this group, send email to ope...@googlegroups.com

> >> <javascript:>.

> >> To view this discussion on the web visit
> >> https://groups.google.com/d/msgid/openpnp/a7311e8e-6cf7-4422-89e1-731237d3e844%40googlegroups.com

> >> <https://groups.google.com/d/msgid/openpnp/a7311e8e-6cf7-4422-89e1-731237d3e844%40googlegroups.com?utm_medium=email&utm_source=footer>
> >> .

> >> For more options, visit https://groups.google.com/d/optout.
> >>
> >
>

> --
> You received this message because you are subscribed to the Google Groups "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
> To post to this group, send email to ope...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/88aa2ec5-a8f1-4e87-ad9b-41608e80ca39%40googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
B.Walter <be...@bwct.de> http://www.bwct.de
Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.

[Daren Schwenke]

No luck.  Tried system paths, common linker paths, and then the local program paths in the root, lib and target dirs with and without the linux-arm directory.

I think it's specifically looking in the jar and nowhere else as the error looks like this:

2018-08-19 12:26:43 SystemLogger ERROR: Exception in thread "AWT-EventQueue-0" java.lang.UnsatisfiedLinkError: Unable to load library 'openpnp-capture': Native library (linux-arm/libopenpnp-capture.so) not found in resource path ([file:/home/pi/Installs/openpnp/target/openpnp-gui-0.0.1-alpha-SNAPSHOT.jar])

[Daren Schwenke]

I'm starting with the develop version which has opencv 3+ so I'm not sure how much of this would apply, but it is still good information for me.  Thank you.

[Daren Schwenke]

I think I just need a lesson in working with something that has been 'mavenized'.

i'm looking through openpnp-capture-java about releases and the part where you can package it back up to submit.  linux-arm doesn't exist there of course and those assume the binary comes from github.

I have everything built here, but how do I tell my local openpnp copy to use what I've built instead for openpnp-capture/openpnp-capture-java and just put that right into the jar I'm building?

[Daren Schwenke]

Ok.  So I duplicated the steps the download script would do, adding my linux-arm binary where it would go.  Incremented the version so I can be sure I'm using the new one.  Packaged it up... builds, works as far as I can tell.

Created a local file based repo from that with:  

mvn deploy:deploy-file -Durl=file:///home/pi/Installs/openpnp/repo/ -Dfile=target/openpnp-capture-java-0.0.13.jar -DgroupId=org.openpnp -DartifactId=openpnp-capture-java -Dpackaging=jar -Dversion-0.13

Changed the openpnp pom.xml file to point at it by changing the openpnp-capture-java dependency incrementing the version to make sure I got mine, and added my local repository:

<repository>
  <id>openpnp-capture-java</id>
  <url>file:${project.basedir}/repo</url>
</repository>

I'm doing something wrong though still, cause when I go to package it up, it fails with missing dependencies.

Failed to execute goal on project openpnp-gui: Could not resolve dependencies for project org.openpnp:openpnp-gui:jar:0.0.1-alpha-SNAPSHOT: Could not find artifact org.openpnp:openpnp-capture-java:jar:0.0.13 in openpnp-capture-java (file:/home/pi/Installs/openpnp/repo)

[Jason von Nieda]

Hi Daren,

First, a few hints about Maven:

1. Maven creates a cache on your drive, usually under $HOME/.m2. That's your local repo.

2. When you run mvn against a project, it will check if any dependencies are missing from your local repo and download them to your local repo. Then the project will use them from that directory.

3. You can "install" a project into that local repo by running `mvn install`. That will build the JAR archive using whatever version is listed in pom.xml and then replace whatever is in the local repo with that archive.

So, with that in mind, here's how I would accomplish what you are trying to do:

1. Check out the openpnp-capture-java project from https://github.com/openpnp/openpnp-capture-java

2. Follow the Build instructions in https://github.com/openpnp/openpnp-capture-java#build to get an initial build.

3. Add in your self compiled linux-arm binary at openpnp-capture/binaries/linux-arm/libopenpnp-capture.so

4. Run `mvn install`. This will build a new JAR and put it in your local repo using the same version as the current release.

5. Check out openpnp from https://github.com/openpnp/openpnp

6. Run `mvn package` which will build openpnp.

7. Run openpnp.sh which will start your self built openpnp. 

So, in English, what is happening here is that you are building a new version of openpnp-capture-java 0.0.12 and replacing the downloaded copy with your own. Then when you rebuild OpenPnP it will pick up that new jar and use it instead of the previously downloaded one.

To verify this worked, if you `unzip -t target/lib/openpnp-capture-java-0.0.12.jar` you should see your linux-arm binary inside.

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/a866d926-faaf-4ca3-badd-ed2feeed49d1%40googlegroups.com.

[Daren Schwenke]

Thank you.  I'll give that a shot and report back.

[Jason von Nieda]

One other note: if you want to get the arm version in the official release we will need to find a way to cross compile it. The release builds are done with Travis CI (for Linux and OS X) and Appveyor for Windows. So we will need to find a way to do that for Pi as well.

Jason

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/c3b781c9-e51a-473a-ac31-e726f7250f50%40googlegroups.com.

[Daren Schwenke]

My cohorts in the Machinekit space would probably know exactly how to do that, as they do it now.  I'll see if someone would be willing to do a little knowledge transfer.  Thank you.

[Brynn Rogers]

I was going to ask on what arm platform your running things, but Jason just implied that it is a Raspberry Pi.

So I'll ask a bit more...    Pi 2,3 ,plus?  or something else?

And while I imagine any Pi is probably fast enough for the general motion,   I am curious about a couple could be bottlenecks...

1) is there only one USB controller, and does that mean only one camera? [at a time?]

2) is the image processing slower on the Pi?    how much?

I haven't picked out a PC to use for my pnp yet.   Probably an a old Dell at the office, size isn't really going to be an issue for my machine  ('cause it is kindof huge.)

Brynn

On Sunday, August 19, 2018 at 4:45:27 PM UTC-5, Daren Schwenke wrote:

My cohorts in the Machinekit space would probably know exactly how to do that, as they do it now.  I'll see if someone would be willing to do a little knowledge transfer.  Thank you.

out.

[Daren Schwenke]

ignoring the general recommendation of NOT using the Pi, I'm using the Pi 3B+.  Any PC will likely outperform the Pi, but I'm trying to keep the barrier of entry as low as possible here and the Pi will work.

Also, openpnp won't start on it unless you turn down the GPU memory and limit the java heap size at runtime.  I also configured swap (for bulding) but I don't think it needs it to run.

If I recall correctly, the simulated vision tests which flew by while I was compiling ran at about 700 parts per hour.  If someone told me how to kick that off again, I will.

I'm not using USB.  I'm using the CSI connector on the Pi.  This eliminates the need for compression/decompression or pre-scaling of the images as it has the bandwidth to just send everything uncompressed, and it eliminates usb latency.  So this lets me get the full resolution image, and I'll be cropping the image locally for bottom vision as my mirror only uses about half the FOV.  I'll probably be using it full resolution (5mp) for the top which might be kinda slow.

You can run two cameras on the single CSI connector, but I'm not doing that.  I'm doing this:

https://hackaday.io/project/45404-arcus-3d-p1-pick-and-place-for-3d-printers

Given how piggy the java was, I also decided I'm not running my motion platform on the Pi.  For now I'm going to be feeding a BBG running Machinekit and using the PRU's there for stepgen/PWM.  

That part is done, minus the software connection to it as I don't like how the LinuxCNC driver works and Machinekit actually exposes a nice networking layer for remote interface support.  A simple client for it is here: https://github.com/machinekit/pymachinetalk
I have yet to do any of that.  We'll see if I have time.

[Daren Schwenke]

The 'go to' guy for the cross platform builds, John, is currently moving so it may be a while.  :(  

I'll still see if the collective remaining knowledge could help get this started though.

[Daren Schwenke]

This worked!  Now I just need to figure out how to make the device show up.  Thank you.

On Sunday, August 19, 2018 at 5:31:22 PM UTC-4, Jason von Nieda wrote:

[Brynn Rogers]

I saw your link already in the talk about fly camera's thread, or somewhere else...

One thing I haven't seen yet is a video of this string delta running.   Do you have any video of the whole thing running?

How do you keep the head from swinging on the strings?   It seems like once disturbed, you would have to wait forever to let any swinging damp out, making the whole thing terribly slow.

Brynn

P.S. my 3d printer is a rostock max V2,  a delta style.

[Daren Schwenke]

https://www.youtube.com/watch?v=KQ19fT4BQQU&list=PLsLsg-CGKqySFfwt2Oluhc0Ci83hq13zR

Video 3 pretty much sums it up.

[Daren Schwenke]

Progress... Realized I turned down the GPU memory too much to load the video driver.  Turned it back up to 128 (I'll find the lower limit later).

Then: sudo modprobe bcm2835-v4l2

and I had /dev/video0

All the native v4l2 tools work.  Overlays are literally realtime.

In openpnp the camera initializes as the light comes on when I start it, but the format menu isn't populated correctly it seems.  All resolutions and framerates are 0.  There are 14 formats though.

I went into machine.xml and tried manually setting the format-id to a couple choices, no change.

Openpnp just steams out a chain of 'java null pointer exceptions'.  So I removed the OpenpnpCaptureDriver and replaced it with 'Webcam' and set a device in machine.xml.  

Same thing, but no idea if this was a valid test of anything.

I do see at startup when I use the console I get:

queryFrameSize returned non-discrete frame size! 14 times.

Since that number matches the number of available formats, I'm assuming that is where this problem is starting, and the rest is just the result.

On Saturday, August 18, 2018 at 10:38:39 PM UTC-4, Daren Schwenke wrote:

[Daren Schwenke]

Well I think I get it.

openpnp-capture/linux/platformcontext.cpp only supports discrete values for width and height from v4l2, but the it looks like v4l2 has two other possible types defined as well.
https://www.linuxtv.org/downloads/legacy/video4linux/API/V4L2_API/spec/rn01re29.html

[Paul Jones]

Capture uses the cmake build system which is quite good with cross compiling. I should be able to get it working without too much effort. I’ll have a try and see how I go.

(Gives me an excuse to buy a pi3 too 😊)

 

Paul.

 

 

From: ope...@googlegroups.com <ope...@googlegroups.com> On Behalf Of Jason von Nieda
Sent: Monday, 20 August 2018 7:42 AM
To: ope...@googlegroups.com
Subject: Re: [OpenPnP] Re: Building on armv8.

 

One other note: if you want to get the arm version in the official release we will need to find a way to cross compile it. The release builds are done with Travis CI (for Linux and OS X) and Appveyor for Windows. So we will need to find a way to do that for Pi as well.

Jason.

[Daren Schwenke]

Sounds like a good excuse as any.  :)  This was my first Pi 3, and whatever happened, this one was destined to run openpnp from the start.  

I went off on a 'hardware overlay' tangent here.  The Pi camera can be directly displayed using the GPU.  Not that useful, except that the Pi also lets you draw directly on the HDMI framebuffer if you want, and so you could display the pi camera output in realtime, and then overlay what opencv wants to draw on top of it.  Well that would work except you would have to draw the overlay every frame then, what you captured wouldnt' exactly line up with what opencv wanted to show, and it might actually be slower as it would skip the GPU.  

Perhaps switch between the two modes.  Display the overlay until opencv needs to do work, then switch it off and show the opencv output.

I don't know... it's a thought...

In any case my plan is go looking for code which handles the other two modes returned here.  If I can't find any, I'll just return the maximum width/height instead for now.

[Daren Schwenke]

Closer now.

Made the hack to add stepwise as an option to the C, just returning max_width/height.  

No more null pointers, the formats now show up under format, brightness control sliders, etc are all there, and the camera initializes.

But then it dies.

Console now complains of 

getPropertyLimits (ID=3) failed on VIDIOC_QUERYCTRL (errrno 22)
getPropery (ID=3) failed on VIDIOC_G_CTRL (errno 22)
Select timeout

Looking around for that...

[Daren Schwenke]

The first two look like what happens when you try to pull a propertyLimit on an unsupported device property. Probably not breaking anything unless it was set to break something...

The select timeout looks like where I'm going to get in over my head here.  

Any takers on working on this with me?

[Niels Moseley]

Hi Daren,

I will help you with this as soon as I get back from my vacation.
Please give me a few days :)

Regards,
Niels.

--

You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/f7b2c87a-15dc-436c-8831-ae87b79e4c2e%40googlegroups.com.

[Daren Schwenke]

Thank you Niels.  I'll continue to plow forward if I can and post updates here, while I await your return.

Noticed the maxFPS was also returning stepwise and not handled, but fixing that didn't help. 

Perhaps I'll try iterating on the possible step values, and adding the valid options so I can select less than max height/width.

[Daren Schwenke]

Well, I figured it out.

Summing up, vfl2 was returning values out of range for the actual hardware, for most everything.

I finally just hardcoded a few known good values in the part that goes hunting for the formats, and now it works fine.

That's pretty sucky, lending itself to not having a dropdown, but instead ignoring what v4l2 is telling you and just having a few text boxes to dump your desired values in.

720p@45fps actual is what I'm getting in case you are curious.  

There is a weird pause every so often, and it's supposed to be running at 60fps, but I'm not complaining.  

I can now move on to other, more broken things.