Incorrect modelfitting with different mesh

64 views
Skip to first unread message

Masu de Ridder

unread,
Sep 29, 2021, 7:45:33 AM9/29/21
to scalismo
Dear Marcel,

This is in addition to the original conversation, but when I reply to those the message gets deleted and therefore I created a 'new' conversation.

I have tried your suggestions but unfortunately, the results have not improved much.
Attached figures are:
- The input mesh
- The result after using the posterior model for the fitting
- The result after using a truncated model for the first 10 iterations, consisting of 3 components, with noise term 10
- The result of parametric non-rigid iterations, which I wanted to try out. However, the face is twisted and I am not sure what causes this

I would really appreciate any further feedback.

Kind regards,
Masu de Ridder
Input.png
fitted_nonP_pasha.png
posterior.png
fit_truncated3.png

Marcel Luethi

unread,
Oct 2, 2021, 2:37:12 PM10/2/21
to Masu de Ridder, scalismo
Dear Masu

The twisted face looks like something really bad is going on. As far as I remember, you wrote that you ignore the mouth cone in the fitting?  If not, this could be the reason.
The other results look better, but the artefacts are not pretty either. It looks like there is an overfitting going on. One strategy could be to either slightly augment the model or make it a bit more local by multiplying the ssm with a Gaussian kernel, such that there is a bit more flexibility.

Could you observe when these artefacts start appearing? Is it already early on in the fitting process or only in the end stage?

Best regards,

Marcel

--
You received this message because you are subscribed to the Google Groups "scalismo" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scalismo+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scalismo/3dc84ad8-ef64-4163-9125-8e25d5c97ea6n%40googlegroups.com.

Masu de Ridder

unread,
Oct 5, 2021, 11:20:13 AM10/5/21
to scalismo


Dear Marcel,

You are correct, for the parametric I forgot the mouth cone and after correction, it did result in a 'normal' face but not an identical one (figure 1).
For the ICP version, I tried the multiplication and it also improved the results, the mouth is a better fit now (figure 2) but still not as identical as I had hoped. Can the shape difference with ICP be explained by the way I define the correspondence? I have placed landmarks on the reference modelmesh and used attribute correspondences to find their corresponding points on the targetface. When using the function nonrigid ICP I assign the mean mesh of the model as the moving mesh. As I believe you did too in tutorial 11.

If you mean the overall shape of the face with the artefacts, this is from the first iteration for non-rigid. For the parametric version, the deformation is happening in the end stage. I tried it with fewer iterations but there was no clear difference in the result.


Kind regards,
Masu de Ridder
fit_localflex.png
fitted_mesh_parametric.png

Marcel Luethi

unread,
Oct 6, 2021, 3:03:15 PM10/6/21
to Masu de Ridder, scalismo
Dear Masu

Face registration is a tricky problem. You will see many problems that you don't notice in other structures, simply because we as humans are so tuned to recognise faces. A lot of care has to be taken in the setup of the process.
Maybe you can get some inspiration from our face registration pipeline:

Best wishes

Marcel


Reply all
Reply to author
Forward
0 new messages