Open PNP software stops suddenly during working

[pham thang]

Hi Jason

I have 4 PCBAs in one panel but can't run it concurrently due to the Open PNP software stops suddenly during working.

And when i run it one by one, nothing happen.

I don't know what is missing in this case.

Could you please help to advise.

Thank you

[Marek T.]

Hi Thang

"stops suddenly" sounds really not precisely... Just stops with not any message thrown? Like when finished placement? Or hangs up?
Probably you have tried with "reset all placement" before placement running...

[Marek T.]

And nothing helpful in the log file? Maybe put it here.

[pham thang]

Hi Marek T

Yes, it stops without any message, (auto close the software ),  When i open the software again but nothing was saved .

Do you need i take 1 video.

Thank you

Vào 20:13:18 UTC+7 Thứ Bảy, ngày 04 tháng 11 năm 2017, Marek T. đã viết:

[Cri S]

I have seen the new software don't display the result of fiducial
matching anymore,
maybe deleting the image is faster as converting it into bufferedImage.
If that seems true (unconfermed) on certain system that could result
into core dump.


add -XX:+ShowMessageBoxOnError to the java command line inside openpnp.bat or
sh if you use linux.

> --
> You received this message because you are subscribed to the Google Groups
> "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openpnp+u...@googlegroups.com.
> To post to this group, send email to ope...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/openpnp/00b7499f-7111-4153-a06e-0eec58c8529d%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

[Marek T.]

But Cri, Thang didn't say when the problem occurs... we don't know it's normally pasisng through fiducials checking and breaks while placement or not.

Thang, I also don't know what you mean telling "auto close the software". Does the software closes completely or stops mounting? If it closes it smells like some java problem maybe?

I think you should try to check what's in log and also run openPNP from the bat as CriS advises. Then you should see some information in console which may be helpful for diagnose.

Video could be probably helpful to see how all it is at you.

[pham thang]

Hi Marek T/ Cri S

Thank you for your advise. My mean that after place some  part, my machine stops mounting,

and the OPEN PNP Software also closes itself. (Missing all of mounting data)

Sometime, the machine under standby also closes itself.

I am thinking the problem belong to my PC, and still follow up

Thank you

Vào 00:01:03 UTC+7 Chủ Nhật, ngày 05 tháng 11 năm 2017, Marek T. đã viết:

[Marek T.]

Hi Thang
Keep us updated when solved this problem. Worth to know this :-).

[pham thang]

Hi Marek

Now we have 2 issues.

1. After running 8 panels (32 PCBAs) and some part on next panel, it happened again.

Machine doesn't move

Software also gone 

When i open it again, all of placing data was lost and can't recovery

2. White balance of camera is not stable

When i start the Open PNP software, i didn't see the white color on the tip of nozzle, it can detects the missing part.

But after running about 3 hours, something is difference, you can see very clear the white color on the tip of nozzle.

It means that the camera can't detects missing part anymore, misunderstand the nozzle tip and component.

Thank you

Thang

[Cri S]

Do you have used the additional arguments ?
For camera, that is normal, and it don't matter as after blur and gray
conversion you run
normalize stage that compensates this. This then gives you a image
like the second
image, and the correct way to deal with is is to use maskCircle with
negative numbers
in order to mask out the nozzle.

Cri.S

2017-11-06 17:52 GMT+01:00, pham thang <phamvant...@gmail.com>:
> Hi Marek
>
> Now we have 2 issues.
>
> 1. After running 8 panels (32 PCBAs) and some part on next panel, it
> happened again.
> Machine doesn't move
>

> <https://lh3.googleusercontent.com/-XHgLE3klCd0/WgCPKaP-kQI/AAAAAAAAAa8/ycsIfSpgm8MpFE489jylq_EdeBz98dkggCLcBGAs/s1600/z816698865791_3b06ad3644a47fe58a385960e4b2c4fe.jpg>
>
> <https://lh3.googleusercontent.com/-EElg0UaSEYE/WgCPUcYdLAI/AAAAAAAAAbA/IibWE-5-k1YKHKF3gEtPNa-rqv_txGBrgCLcBGAs/s1600/z816696236243_dbef00d04f0551d5a1766ee4448f70e2.jpg>

>
>
>
>
>
> Software also gone
> When i open it again, all of placing data was lost and can't recovery
>

> <https://lh3.googleusercontent.com/-8qh-MrGTtIQ/WgCP9z1eNUI/AAAAAAAAAbI/nvkhaYGDp24SvPnZTjouBzgAZZueRZhkgCLcBGAs/s1600/z816696785134_6779e1da3dcff8606d47ba887cf3657f.jpg>

>
>
>
> 2. White balance of camera is not stable
>
> When i start the Open PNP software, i didn't see the white color on the tip
>
> of nozzle, it can detects the missing part.
>

> <https://lh3.googleusercontent.com/-JB_Z5n2UKKg/WgCQ6wSVYWI/AAAAAAAAAbc/EbY_7PZW6OId9FnLKpOfiEK55wIHVTUJgCLcBGAs/s1600/z816696513459_4d523081b308a6ceb18c8cdf72e1bcf8.jpg>

>
>
>
> But after running about 3 hours, something is difference, you can see very
> clear the white color on the tip of nozzle.
> It means that the camera can't detects missing part anymore, misunderstand
> the nozzle tip and component.
>

> <https://lh3.googleusercontent.com/-0J0XlqwlsvQ/WgCRblWi46I/AAAAAAAAAbs/bUAfomJlFAM052nuRXeXP3b763VFVfYggCLcBGAs/s1600/z816696345265_e83848e8d9ce4b2f6b60c4a38a8a37bb.jpg>
>
> Thank you
> Thang

>
> --
> You received this message because you are subscribed to the Google Groups
> "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openpnp+u...@googlegroups.com.
> To post to this group, send email to ope...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/ca0cb677-69d4-4b6b-a8b6-c325f19f0a95%40googlegroups.com.

[Bernd Walter]

On Monday, November 6, 2017 at 5:52:59 PM UTC+1, pham thang wrote:

Hi Marek

Now we have 2 issues.

1. After running 8 panels (32 PCBAs) and some part on next panel, it happened again.

Machine doesn't move

 

I don't know where to look with windows, but operating systems may kill processes when running out of memory.
Modern OS all overcommit memory.
Same for processes, when a non optional memory request was declined by the OS, they often kill themself.
I've had several "crashes" with OpenPnP, but only with partial functionality in the GUI, never a full termination and I think such a problem is very unlickly, since OpenPnP catches all kind of possible problems.
But if you run out of memory or into per process memory limits then it is out of control.
It can also be a crash in some of the libraries, which don't catch all kind of problems, but usually you get a crash dialog from Windows in that case.

[Marek T.]

Hi Thang,

Firstly you must know I'm not an expert of Java, you should rather follow what CriS advises. Also Bernd's suggestion about memory may be valuable.
However strange that you had not that effects with previous version of Openpnp.
Any problem to back up to the previous version and make sure is it Openpnp or machine/system?

Using Xms/Xmx you may try assign more memory to java. However I'm not sure if it's still working in Java8, you must read about it or just try as a test sample string like below (put it to the i.e. openpnp.bat file).

D://"Program Files/openpnp/jre/bin/java" -Xms1024m -Xmx2048m -jar D://"Program Files/openpnp/openpnp-gui-0.0.1-alpha-SNAPSHOT.jar"

to run message box that CriS mentioned:

D://"Program Files/openpnp/jre/bin/java" -XX:+ShowMessageBoxOnError -jar D://"Program Files/openpnp/openpnp-gui-0.0.1-alpha-SNAPSHOT.jar"
or
D://"Program Files/openpnp/jre/bin/java" -Xms1024m -Xmx2048m -XX:+ShowMessageBoxOnError -jar D://"Program Files/openpnp/openpnp-gui-0.0.1-alpha-SNAPSHOT.jar"

sorry if it's obvious for you and you tried it already...

[Cri correct it if I'm mistaken]

[pham thang]

I am thinking why only me encountered this issue.
I am considering to change PC OS. I am using window 10, maybe Win 7 or linux is better.
Could someone tell me which one is better.

Thank you

[Marek T.]

Thang, You combine too much instead of simple tests. Have you tried what advised by Cri?

Hmmmm.... My oppinion about W10 is none but wrong.
Probably Linux could be best but I'd need learn from begining all the tools required to live - don't have a time for it.
I'm using W7-64b Ultimate and have no problems.

I have had a problem like you say but it was problem of Java not updated by Jason enough. It's old case but maybe something similiar at you.
You can try install the most actual Java taken from Oracle for your system and run Openpnp from bat.file using installed Java instead of included by Jason. Something like:
C://"Program Files/Java/jre1.8.0_141/bin/java" -jar C://"Program Files/openpnp/openpnp-gui-0.0.1-alpha-SNAPSHOT.jar"
Of course make proper locations and version number. I think it's worth to try.

[Marek T.]

Hi Thang,.
My soft crashes too. Not the same like your but maybe it's the same reason. So don't worry it's only your problem :-).

Hi Jason,

Following problems with new soft:
- checkboxes "placed" are not set if you choose "check fiducials". Placed parts' fields are going into green properly.
If fiducials are set off - checkboxes "placed" is set properly after placement.

- when run the panel containing 6 boards with ~50 parts/board - soft "not responding" after fiducials checking. CPU usage by Java increases from standby openPNP 75% to 99-100%. Note that I have A8-3.7GHz processor with 8GB of RAM, so machine is rather good. Using your Java not installed separately.
If I limit this panel to 2-3 boards it goes after fiducials checking normally, but after when head has finished the placement of picked parts, soft freezes at next>place>plan for 3-10sec (it's not constant time, seems as less parts left to place the pause becomes shorter).
If I run this panel with all 6 boards but with only one part on each - this pause doesn't occur and everything perfect.
So as more parts to do then appears cpu problem and pauses.

The previous soft from Oct.22 doesn't have this problem with "not responding" and uses 10% of CPU less. "Dependency" placed-checkboxes to fiducials-checkboxes identical wrong like in last version. You can easy reproduce it.

BR
Marek

[Michael G.]

hi marek,

at least the one issue you have is in focus: https://github.com/openpnp/openpnp/issues/663

[Cri S]

The placed is a bug, i have telled it shortly after commit and the
response was, it was
tested and works, the bug remained, i suppose the test was lacy.
As this "placed" code is a big issue and it interfere to component
skip/repeat code, it
is a big red issue for me how to handle it. Hovewer the placed is
separate and with high
probility it don't have to do with any failure i think. If not, with
the flags for java that i have
suggested there must be a reasonable null pointer exception with named
varaibles that
clearly identify this problem,. I don't think it is this, as java
generally handle null object access
nicely,

> --
> You received this message because you are subscribed to the Google Groups
> "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openpnp+u...@googlegroups.com.
> To post to this group, send email to ope...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/386ea095-e458-425a-ae60-688e4a81f9bd%40googlegroups.com.

[Marek T.]

Thanks Michael, I've not noticed it was reported by Karl. Have effect exact as he described, fids enabled = placed checkboxes disabled, and vice versa.
So Jason if you'll need I also may deliver you my files to reproduce it if you still haven't do.

About freezing and cpu usage (+plus RAM consumption 5Gb instead of 2.5) just have found it's somehow related with my machine.xml. Taken older one from an update before and it works ok with newest version. I'll compare them tonight.
So Thang - maybe you should try the same if you have older machine.

[Marek T.]

Ok Cri, it's moreless clear about fids that need to be tested/updated yet.

Some idea why after fids checking my cpu goes to 100% and ram from 2.5 to over 5? I know already it's because of machine but where...

[Jason von Nieda]

This is because of the Cartesian product used in planning. Larger jobs eat up a ton of cpu and memory because of this. I think an issue was filed but I can’t check right now. The fix is to simplify the planner, or cache the result. For the time being, if you place less boards at once it should stop happening.

If someone wants to look at this I’d appreciate it. I am swamped at the moment and would not be able to check it for at least a week.

Search the list for Cartesian product.

Jason

On Wed, Nov 8, 2017 at 8:53 AM Marek T. <marek.tw...@gmail.com> wrote:

Ok Cri, it's moreless clear about fids that need to be tested/updated yet.

Some idea why after fids checking my cpu goes to 100% and ram from 2.5 to over 5? I know already it's because of machine but where...

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/232ddf94-3f10-43c2-94db-e2eb59e069c5%40googlegroups.com.

[Cri S]

if you do hough circle checking, it is the same ?

> --
> You received this message because you are subscribed to the Google Groups
> "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openpnp+u...@googlegroups.com.
> To post to this group, send email to ope...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/232ddf94-3f10-43c2-94db-e2eb59e069c5%40googlegroups.com.

[Cri S]

Are you sure that is this ? if so i can change it.

[Jason von Nieda]

I’m not 100% sure that this is the reason for this particular crash, but it is the most likely given the evidence. The same thing happened with a previous user.

The symptom should present as an out of memory error in error.log.

Jason

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/CAJGcfUgELXK5thzD1LbZADkkR-gOGPYRe8u50DjYk3MH0aCyrg%40mail.gmail.com.

[Marek T.]

As I said,
- If I make a panel with six boards and only one part on each - it's ok
- if I make a panel containg 3 boards 50 parts on each board - ok with fiducials (no cpu and ram overusage) but visible pauses after each next>place>plan. Around 10sec at begining of assembling when 150 parts waiting for pnp, and still faster as less part remain to place on the list. At the end is perfect.
- if I make a panel containg 6 boards 50 parts on each board - CPU 100%, RAM goes slowly up from 2.5 (standard) to abnormal over 5GB and soft "not responding".

But when taken older machine it is ok with fiducials. Unfortunately can't use old machine directly because made there some changes with tuning. Just now I'm trying to compare them and find what's suspected.
I can put then here if it may be interrested to find a reason.

[n.a.m...@gmail.com]

Hi Jason,

Can you give a short description of how the Cartesian planner works. It suspect it has O(N^2) complexity, is that correct?

Regards,
Niels.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/CA%2BQw0jwZes6cSJ-Gbj2EYj6JLgBRRKstN_TjtgarHf%3DidG4_OQ%40mail.gmail.com.

[Jason von Nieda]

Yes, that sounds exactly like the Cartesian product bug. I'm not sure why it would change on an older version of the code, as the planner has not changed in a couple years. Can you verify for me that you get different behavior on the exact same job and same machine.xml - the only difference being the version of software?

If so, the cause is most likely just different memory availability. 

Jason

[Marek T.]

Ok, so I've found following main differences between my old and new machines (the rest are calibrations, feeders etc). Old works without "not responding" after fids when many parts to serve and no exceeds cpu/mem.
visual homing in both gcode drivers have enabled in both gcode drivers, and
enabled-averaging="false" repeat-fiducial-recognition="3" in top camera.

what is visual homing and how works enabled-averaging ? Was about it on the group? I'm really trying to read everything...

----------------------------
      <driver class="org.openpnp.machine.reference.driver.GcodeDriver" port-name="COM5" baud="19200" flow-control="Off" data-bits="Eight" stop-bits="One" parity="None" set-dtr="false" set-rts="false" units="Millimeters" max-feed-rate="50000" backlash-offset-x="0.0" backlash-offset-y="0.0" non-squareness-factor="0.0" backlash-feed-rate-factor="0.0" timeout-milliseconds="5000" connect-wait-time-milliseconds="1000" visual-homing-enabled="true" name="GcodeDriver">

      <driver class="org.openpnp.machine.reference.driver.GcodeDriver" port-name="COM5" baud="19200" flow-control="Off" data-bits="Eight" stop-bits="One" parity="None" set-dtr="false" set-rts="false" units="Millimeters" max-feed-rate="50000" backlash-offset-x="0.0" backlash-offset-y="0.0" non-squareness-factor="0.0" backlash-feed-rate-factor="0.0" timeout-milliseconds="5000" connect-wait-time-milliseconds="1000">
----------------------------
            <gcode-driver port-name="COM3" baud="115200" flow-control="Off" data-bits="Eight" stop-bits="One" parity="None" set-dtr="false" set-rts="false" units="Millimeters" max-feed-rate="60000" backlash-offset-x="0.0" backlash-offset-y="0.0" non-squareness-factor="0.0" backlash-feed-rate-factor="0.0" timeout-milliseconds="10000" connect-wait-time-milliseconds="1000" visual-homing-enabled="true" name="GcodeDriver">

            <gcode-driver port-name="COM3" baud="115200" flow-control="Off" data-bits="Eight" stop-bits="One" parity="None" set-dtr="false" set-rts="false" units="Millimeters" max-feed-rate="60000" backlash-offset-x="0.0" backlash-offset-y="0.0" non-squareness-factor="0.0" backlash-feed-rate-factor="0.0" timeout-milliseconds="10000" connect-wait-time-milliseconds="1000">
----------------------------
      <fiducial-locator class="org.openpnp.machine.reference.vision.ReferenceFiducialLocator" enabled-averaging="false" repeat-fiducial-recognition="3">
      <fiducial-locator class="org.openpnp.machine.reference.vision.ReferenceFiducialLocator">
----------------------------

[Jason von Nieda]

Hey Niels,

In brief, for every placement cycle we calculate the cartesian product of placements + 1 and nozzles + 1. This is then sorted and filtered to produce the planned job placement that will be used for that cycle. In general, I would not expect this to be overly taxing for any modern computer, but I have not had a chance to really jump in and see what is happening.

One thing worth checking would be if there are any logger lines that are outputting the results. That would be incredibly slow. I feel like maybe there might be one when TRACE is turned on.

The code to look at is ReferencePnpJobProcessor#doPlan(). 

My suggestion would be to load up a profiler, create a job with 50 placements and 6+ boards and run a placement. YourKit profiler is free to try and is very good - it's what I use. 

Here is the other thread where this is discussed, and some potential solutions are proposed: https://groups.google.com/d/msgid/openpnp/0cb4a3a4-95b6-4f4a-b046-1dfbe059d276%40googlegroups.com?utm_medium=email&utm_source=footer

Jason

[Marek T.]

I've run previous version with machine before update and was ok with the same job.
Moved machine from older version to the newest version and was also ok - but only about fids and cpu/mem exceeding (not occured). Haven't time to check with after next>place>plan latencies.
Previous version doesn't started with after_update machine because of some changes in gcode driver parameters and I've left it as had leave an office.

[Marek T.]

W dniu środa, 8 listopada 2017 16:56:11 UTC+1 użytkownik Jason von Nieda napisał:

Hey Niels,

In brief, for every placement cycle we calculate the cartesian product of placements + 1 and nozzles + 1. This is then sorted and filtered to produce the planned job placement that will be used for that cycle. In general, I would not expect this to be overly taxing for any modern computer, but I have not had a chance to really jump in and see what is happening.

One thing worth checking would be if there are any logger lines that are outputting the results. That would be incredibly slow. I feel like maybe there might be one when TRACE is turned on.

It was definitely OFF. Later I've turned it on to see that delays logged.

The code to look at is ReferencePnpJobProcessor#doPlan(). 

My suggestion would be to load up a profiler, create a job with 50 placements and 6+ boards and run a placement. YourKit profiler is free to try and is very good - it's what I use. 

I can put here tomorrow my concrete job if it may be usable.

[Bernd Walter]

On Wednesday, November 8, 2017 at 4:03:36 PM UTC+1, Jason von Nieda wrote:

This is because of the Cartesian product used in planning. Larger jobs eat up a ton of cpu and memory because of this. I think an issue was filed but I can’t check right now. The fix is to simplify the planner, or cache the result. For the time being, if you place less boards at once it should stop happening.

If someone wants to look at this I’d appreciate it. I am swamped at the moment and would not be able to check it for at least a week.

I wanted to write and test another logic, but getting my machine running still has higher priority for me and it only happened to be a problem with many valid nozzles.
But I'm not very convinced that this is Phams problem at all.
As far as I can tell from the vidoes he is currently running with a single nozzle.
And he triggers his problem by running a panel with multiple board, but aren't the board run one after the other?
So this wouldn't this lead to running n boards with the same number of components one after another, or does OpenPnP run all boards at the same time?
If the later then of course it puts more parts into the planer at once, but still with a single nozzle.

Marks situation sounds more like this and he could easily work around by not allowing multiple loaded nozzletips for the same part.

[Cri S]

Multiple boards are run at same time, not sequencially.

> --
> You received this message because you are subscribed to the Google Groups
> "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openpnp+u...@googlegroups.com.
> To post to this group, send email to ope...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/eb2230ea-a617-48da-b6dc-60e53a7e6436%40googlegroups.com.

[Jason von Nieda]

All boards in a job, whether in a panel or not, are planned at once for every cycle. This is required to optimize nozzle changes.

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/eb2230ea-a617-48da-b6dc-60e53a7e6436%40googlegroups.com.

[Marek T.]

Hi Jason, Cri

Jason of course I've found what is visual homing and averaging, great option the last one as for me.

About my problem, I've found what in the machine makes trouble but really don't understand why. It's the only one line 224 containing <id>0402_G</id> for one of tips at Nozzle3. If I disable it the soft works normally. When finished with fiducials I guess Openpnp calculates the plan and there is small pause 2-3sec then goes further. While the calculation CPU usage grows from some 85 to 92% if without this 0402_G. If this 0402_G is turned on then CPU usage grows about 15% from 85 to 99% and "soft not responding" (waited few times 1-2 minutes for responding but nothing).
Definitely this new version eats some 5-10 more CPU than previous. At previus version CPU usage grows also for 15% while calculation but from 75-90 and no exceeds 100%. Maybe here is a reason of problem, but why the base consumption increased 10% between versions? Also question with woth this 0402_G the usage grows so much? Some idea what to do, optimize by me in machine or by you in the code?

Attaching both machines, pls try to look on it in free time.
At occasion I'd like ask for my mess explanation. In few places ID of my nozzles is like <id="NT2" name="501">  but in other <id="TIP1505293224094" name="502">. Names are made by us in GUI but what about the ID? We have not edited them, they are as are on 'theirself". Should I edit them manualy, how recommended to do it well?

thx
BR
Marek

[Marek T.]

Update:
It's not related with tip on nozzle 3 concretely. Just in this job there is a lot of parts to be pnp as 0402_G so it means a lot of power for plan calculation. If I disable this compatibility for any other nozzle tip it also works ok. So the generaly it's the problem of power for plan calculation.
btw: I had set high refreshment for cameras 30fps. So lowered it down to 10fps and standby openpnp cpu usage fallen to 60%. But when comes to the plan calculation it grows to 99-100% and dead.

W dniu środa, 8 listopada 2017 17:54:13 UTC+1 użytkownik Jason von Nieda napisał:

[Marek T.]

Update2:
I have checked what you asked me to confirm. Ran previous soft with job and machine taken from last version (removed only declarations of visual homing and averaging to could run it at all). The same behaviour :-(. So it's some general problem not problem of last version only. Sorry for confusing but problem still remains. really 6x50 parts is not some big project... Also lowered resolution to 1fps and standby consumption 510% grows to 100% while the plan calculation with my 0402_G "many" parts.

W dniu środa, 8 listopada 2017 17:54:13 UTC+1 użytkownik Jason von Nieda napisał:

[Jason von Nieda]

Thanks marek, that confirms my suspicion. This is the Cartesian product bug. I will work on a fix in one week. Hang tight and in the mean time just place one board at a time if you can.

Jason

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/60f57161-e96b-46a6-8aa6-dbc73a8b4e3b%40googlegroups.com.

[Marek T.]

So excellent that you already know where the problem is hidden :-).
If you will have something to test I can check it within one day on my machine.
At occasion, maybe it's not worth to touch or maybe worth to update, but current Java version is 151 while your install packet contains 131 as I think.

[Jason von Nieda]

Good point Marek, it's high time I update Java. I will do that soon. 

Jason

[Cri S]

I have now timed a simpler alternative planning algorithm without
changing that many classes, and on 125235 components
(1089 boards into one panel) it needs 15.8 seconds into preFlight
stage for. doing initial
planning. It don't seems that efficient at all. As it uses internal
list that are planned at initial
state, if started components cannot be removed during job run.
Maybe it is worth waiting for Vonnieda optimisation that don't have
this side effects.

>>>>> <https://groups.google.com/d/msgid/openpnp/eb2230ea-a617-48da-b6dc-60e53a7e6436%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>> .

>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>> --
>>> You received this message because you are subscribed to the Google Groups
>>>
>>> "OpenPnP" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>>

>>> email to openpnp+u...@googlegroups.com <javascript:>.

>>> To post to this group, send email to ope...@googlegroups.com

>>> <javascript:>.

>>> To view this discussion on the web visit

>>> https://groups.google.com/d/msgid/openpnp/60f57161-e96b-46a6-8aa6-dbc73a8b4e3b%40googlegroups.com
>>>
>>> <https://groups.google.com/d/msgid/openpnp/60f57161-e96b-46a6-8aa6-dbc73a8b4e3b%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .

>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openpnp+u...@googlegroups.com.
> To post to this group, send email to ope...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/b018c789-4eb2-4b25-9bda-c6b034f3c15e%40googlegroups.com.

[Marek T.]

125K, wow, impressing. I'm dying in 3sec with 416x less components ;)
Yes, let's wait, better to have official optimisation made once and good.

[Jason von Nieda]

Okay, I've verified this problem. It really comes into focus when there is more than one nozzle. Since I do most of my testing with one nozzle, it doesn't come up often.

I tested with 4 nozzles using a job with 900 boards of 200 placements each, and that shows the problem quite nicely :) All 8 cores of my machine have now been pegged at 100% CPU for a couple minutes. Maybe it will finish someday!

And... it just ran out of memory. So yes, major issue.

Job and config are attached if anyone would like to play around with this.

I'm open to ideas on how to best fix this. Cri's method, which plans ahead of time and then caches the results will work up to a point, but I fear that for very large jobs it will fail in the same way.

Over the next few days I will play around with some alternate planners. I'm thinking of just simplifying it quite a bit so instead of trying to find the "best" solution, we just find the nearest solution. This may result in more nozzle changes overall, but I think we can optimize for that a bit just by sorting, rather than creating the cartesian product. 

The purpose of the cartesian product is to try to find ideal solutions, but for a job this large that just might not be feasible.

Jason

[Cri S]

No, it works. Hovewer my algorithm need to cache the job.
No further modification of job is possible after job have started
until job ends.
It is possible, it is used on the next job, not the current one.
I do different planning initially, basically i group placements orders.
You simple sort on placements height and then start from smalles and
place it basically
in that order.
I divide placements that need to be placed after / before the other
placements, resulting
in a sequence of groups of placements. There are pro and cons to both
approaches.

For nozzle planning, i simple do:
from time to time {
shuffle-placement-list (only that placement that are placeable
with current nozzle tips)
sort-placement-list based on location of first list entry
}
for(i=0;i<noz.size();i++)
planned.add(getNextPlacement(noz.get(i).getNozzleTip()));
for(i=0;i<noz.size();i++) if(planned.get(i)==null)
planned.set(i,getNextPlacement(noz.get(i)));
for(i=noz.size();i--;) if(planned.get(i)==null) { noz.remove(i);
planned.remove(i); }

[Marek T.]

Jason, it's excellent that you could reproduce it on your machine!
Hope you will find a solution together :-)

[Marek T.]

Hi Jason,

Pls don't take it as some pressure, I guess you work on few different problems simultaneously...
In the end of this week have planned some larger job with many parts and multi-nozzle using...
Have you found some solution for this problem or I rather must divide the job into few smaller parts...?
Just let me know pls.

br
Marek

[Jason von Nieda]

Hi Marek,

This is not likely to be fixed by the end of the week. One thing to consider is that if you just disabled some boards from the job, this decreases the workload. You can just uncheck some of the boards, process the job, then uncheck the ones that are done and check the ones that are not and run it again. 

This bug is my highest priority right now, I just need to find some time to fix it :)

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/540ce338-a609-4f87-bafd-62709dd116c3%40googlegroups.com.

[Bernd Walter]

On Wednesday, November 29, 2017 at 2:18:20 AM UTC+1, Jason von Nieda wrote:

Hi Marek,

This is not likely to be fixed by the end of the week. One thing to consider is that if you just disabled some boards from the job, this decreases the workload. You can just uncheck some of the boards, process the job, then uncheck the ones that are done and check the ones that are not and run it again. 

You could also try the attached diff.
Just finished it minutes ago, but had no time to test it on real hardware yet.
It builds however and the build tests run fine at least.
The build tests are actually really good and catched all expected failures when building intermediate code.
I don't consider this as a complete generic solution as it doesn't even try to reduce the number of nozzle tip changes.
So if you use a changer or even change tips manually, this will not be for you as is.
It also doesn't try to produce the longest list of most nozzles used as the original code does and the result might be a few more cycles.
I'm not changing nozzles on my machine and therefor I'm not entirely sure how it works.
My machine also has feeders spread all over around and my own final needs are different as filling the nozzles with parts from nearby feeders likely would be a more useful criteria to me than maxing the nozzle usage.

[Marek T.]

Hi Jason,

Thx for update!
(There is some little chance that I will know how to select/unselect the boards from the job to decrease the load... ;) )


Hi Bernd,
Have there many nozzles changing in that project to assembly, but thx.

br
Marek

[Jason von Nieda]

Hi Marek,

I just mean the "Enabled?" checkbox in the boards list. Or, if you are using panels, you can use the X-out feature.

Jason

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/f40553da-26a2-4e39-9e7a-456f3650aa00%40googlegroups.com.

[Marek T.]

Hi Jason,

I thought that understand you and let to myself for the small joke, sorry :-).
But now I'm not very sure if I was right...

If I use the board auto-panelized into (i.e. 2x2) "Enabled?" checkobx works only for rirst row and enables/disables whole the panel only, no possible with this checkox to select any single board.
So I use "skip certaines PCBs on Panelized boards", I guess it's the "X-out feature".
correct?

br
Marek

[Jason von Nieda]

Yes, that's correct.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/6c27a8cc-6cd9-4d63-bf40-7433ce3ee46e%40googlegroups.com.

[Marek T.]

ok, so I'm that's still doing.

[Marek T.]

Hi Cri,
Downloaded it but tomorrow I can only check if it overloads the system like current official branch planner.
How it works with real placement I will be able to check in next week only.
So probably Jason will give a more complex feedback much faster...

Anyway, great that you gave uncompiled code as I must put it to my modified version.
Thanks!

br
Marek

W dniu czwartek, 30 listopada 2017 18:27:09 UTC+1 użytkownik Cri S napisał:

Compiled Version if you want test it, including decompiled code (to
fit the rule of one command per line)
for the planning solution.

https://drive.google.com/drive/folders/1XCrBtyfLhMWVJtuYYONrMfMEn4tPLOQu?usp=sharing

In short. if full-calc is set, it checks all parts, otherwise just the
first pleacable part is used.
If middle is negative, both things accessible from job menu, then
instead of height
order for big parts, the placement distance from zero machine is used.
I have removed unsafe boolean, as it is nearly the same as setting
middle to a high value.



2017-11-30 11:30 GMT+01:00, Cri S <phon...@gmail.com>:
> I have a algorithm working with  @Vonnieda "on the fly calculation" .
> Because this on the fly calculation, the hirarchy dependence that is
> based on height
> and distance cannot be performed because it is relative time
> intensitive and not suitable
> for on the fly calculation.
> Instead two parameters are set, "unsafe" (boolean) and middle (height, mm)
> that
> influence the placing process. Currently i don`t have wizard code
> written for it, it is just
> accessible by scriping in case of tweaking needs.
> The code is approx 100 lines ( approx 400 line with single line statement )
> The algorithm is this:
> it groups the placement in 3 category based on the height.
> small middle big. Big are mounted on height basis (current sorting order).
> The other two are sorted in some way based on distance.
> Actually for the big assembly list supplied in this thread,
> it need 89ms for the calc, and after having placed 300 parts, the
> average is 18ms.
> Based on this timing, it should find the nearest part instead of just
> the first. This is actually
> not implemented.
> unsafe is that it can place big parts, if no small or middle parts are
> found for placing on this
> nozzle. Small and middle always have precedence to big.
> Without unsafe, it places the smaller parts, and then places the
> bigger parts above the
> middle height setting. If middle is 0 or <=3mm, middle code is deactivated.
> Maybe this could be changed to allow place parts with >0.6 or 1.0 mm
> into middle queue.
> Precedence is small, middle, big for safe processing and unless all
> small parts are placed,
> no big parts can be processed. Middle and big is ok.
> For unsafe operation, the precedence is just small, middle, big.
> There are always two queue,
> small - middle - big  for current NozzleTip and then
> small - middle - big  needing Nozzle change and precedence always are
> in this two levels.
>
> I don|t know if this type of algorithm is wanted, and it need some
> thinking  for the (un)safe
> placing and selecting threshold for middle height.
> I`m using similar code, but with hirarchical dependency in order that
> the safe/unsafe is
> automatic handled and better automatic optimizing possibility, both
> this needs preprocessing of planning queue and implies that the
> placements cannot be changed after
> starting the job.

>>>>>>>>> <https://groups.google.com/d/msgid/openpnp/540ce338-a609-4f87-bafd-62709dd116c3%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>>>>> .

>>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>>
>>>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "OpenPnP" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send
>>>>>>
>>>>>> an email to openpnp+u...@googlegroups.com.
>>>>>> To post to this group, send email to ope...@googlegroups.com.
>>>>>>
>>>>> To view this discussion on the web visit
>>>>>> https://groups.google.com/d/msgid/openpnp/f40553da-26a2-4e39-9e7a-456f3650aa00%40googlegroups.com
>>>>>>

>>>>>> <https://groups.google.com/d/msgid/openpnp/f40553da-26a2-4e39-9e7a-456f3650aa00%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>> .

>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups
>>>>
>>>> "OpenPnP" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an
>>>>

>>>> email to openpnp+u...@googlegroups.com <javascript:>.

>>>> To post to this group, send email to ope...@googlegroups.com

>>>> <javascript:>.

>>>> To view this discussion on the web visit

>>>> https://groups.google.com/d/msgid/openpnp/6c27a8cc-6cd9-4d63-bf40-7433ce3ee46e%40googlegroups.com
>>>>
>>>> <https://groups.google.com/d/msgid/openpnp/6c27a8cc-6cd9-4d63-bf40-7433ce3ee46e%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>> .

>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "OpenPnP" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to openpnp+u...@googlegroups.com.
>> To post to this group, send email to ope...@googlegroups.com.
>> To view this discussion on the web visit

>> https://groups.google.com/d/msgid/openpnp/464f28cb-f3e9-4098-9227-0f6680b5405e%40googlegroups.com.

[Marek T.]

Hi Jason,

The messages with links which Cri put here are canceled. I don't know by whom, no matter, but think you saw that code of his planner.
Wanted only to tell you that tested them on my project and it's really promising, OpenPnp doesn't overloads CPU when starts the planning and normally goes further to perform assembling at once.
I can't to tell how optimal is this planner algorithm comparing to your one but the system normally goes through whole mounting process. More practical tests will do on these days and let know how it's good.

Unfortunately his code cannot work if AutoChanger flag is not enabled. So is not usefull for those who wants to change nozzles manualy or have fixed nozzles.
I don't need regular AutoChanger for my machine, so for my use I have modified RefferenceNozzle - now when the flag is OFF the changing is not performed at all (like in OpenPnp before manual changing feature applying), when the flag is ON - it is ManualChanger mode (mode like in official branch when the flag is OFF). So now it doesn't realize regular AutoChanger mode at all - good for me but not good to be universal for everybody
The "problem" is a lack of extra flag in GUI to can choose: changerOFF, changerAUTO, changerMANUAL.

After private discussion with Cri I know there is still lack of few other improvements of algorithm that could be done to make it really perfect.

So If you don't have some clear idea how to improve your planner - Cri's idea is worth to consider but requires also a lot of efforts and polishing :-(.

br
Marek

PS. The code which Cri has published here had some small mistake inside. So if you'd like to use it anyway - better contact to him before.

[Jason von Nieda]

I've filed an issue to track this fix: https://github.com/openpnp/openpnp/issues/686

Jason

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/04697283-3d9a-4da9-872c-bd7380fd8bb1%40googlegroups.com.