Shape model struggles

[fifa king]

 When trying to turn the dataset into a set of deformation fields, I get the following error: "java.lang.IndexOutOfBoundsException: 98880.."

here is a segment of the code:

val refLandmarks = pointIds.map{id => Landmark("L_"+id, reference.point(PointId(id))) }

val alignedSet = toAlign.map { mesh =>    

     val landmarks = pointIds.map{id => Landmark("L_"+id, mesh.point(PointId(id)))}

     val rigidTrans = LandmarkRegistration.rigid3DLandmarkRegistration(landmarks, refLandmarks)

     mesh.transform(rigidTrans)

}

val defFields :IndexedSeq[DiscreteVectorField[_3D,_3D]] = alignedSet.map{ m => 

  val deformationVectors = reference.pointIds.map{ id : PointId =>  

    m.point(id) - reference.point(id)

  }.toIndexedSeq

  DiscreteVectorField(reference, deformationVectors)

}

Any ideas?

Thank you!

[Marcel Luethi]

Hi,

Please be so kind to the members of the mailing list to reveal your
name. This makes the interaction much more pleasant for all the
parties involved. Our goal is to build a professional community around
scalismo. For this it is really important for people to be able to
relate to each other.

Regarding your problem:
You should get a stack trace with the exception. The stack trace tells
you exactly in which line the error happens. It should already give
you a hint where the problem lies. If you still cannot find it out,
please post also the relevant part of the stack trace to the mailing
list. This makes it easier for us to spot the problem.

Can you also check if all the meshes you are working with in this code
snippet have the same number of points
(mesh.pointsSet.numberOfPoints). If this is not the case this could
explain your problem.

Best regards,

Marcel

> --
> You received this message because you are subscribed to the Google Groups
> "scalismo" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to scalismo+u...@googlegroups.com.
> To post to this group, send email to scal...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/scalismo/25601a55-6ee8-494e-b51f-50cbe02212dd%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Ryan Lloyd]

Hello,

Thanks for the reply. Okay so the problem is strange. I create posterior shape models and use the iterative closest points method to ensure that my meshes all have the same number of points and are in correspondence with each other. When using the "mesh.pointsSet.numberOfPoints" function, all of my meshes then have the same number of points...(98889 points). Yet when I save the meshes to a folder on my computer, then they no longer have the exact same number of points when imported later on. For instance one would have 98880 and the other 98885 etc. Is this a common problem where meshes lose some points(data) when saved to a folder on the PC? Obviously now, I can't make a SSM on this data where the number of points are inconsistent.

Thank you :)

Ryan Lloyd

[Marcel Luethi]

Hi Ryan,

Thanks for the detailed problem description.

Is my assumption right that you are using stl format to save the
meshes? We recently discovered that there can be issues with this
format. If two vertex points are very close together, it can happen
that these points are merged by the reader. This happens because in
the stl format the vertices are only implicitly defined by the corners
of the individual triangles.

To find out whether this is your problem, you can save the meshes in
.vtk format (MeshIO.writeMesh("filename.vtk"))

In case this is really the problem, you should also consider
downsampling your reference mesh before registration. It would mean
that you have way too many vertices and can reduce them (we usually
use MeshLab of Paraview for this task). Having fewer vertices greatly
improves performance of all modelling task and also reduces memory
usage.

I hope this will resolve your problem.

Best regards,

Marcel

> https://groups.google.com/d/msgid/scalismo/d55ddfa5-722f-4ade-bee8-1d780d855450%40googlegroups.com.

[Ryan Lloyd]

Hi Marcel

Saving as a .vtk file worked! Thanks so much!

Ryan

[Rob Gathercole]

Hi all,

I'm experiencing similar problems but i have confirmed that the meshes I'm using have very different number of points. I see above that you mention 'mesh.pointsSet.numberOfPoints' to ensure that all meshes have the same number of points. I've tried a number of different variants but haven't managed to figure out how to achieve this and so i'm wondering if anybody able to provide an example of how this would look in code? Also, is it better to reduce the number of points or increase them? The meshes in my dataset range from ~5000 points to ~35000. 

Any help would be gratefully received!

Cheers,

Rob

[Marcel Luethi]

Hi Rob,

That your meshes have a very similar number of points indicates that
you have not yet registered them.

Before a shape model can be built, registration has to be performed to
bring all points of the meshes into correspondence.
For this you pick one surface of your data set as a reference and
perform the registration to all the other surfaces in the data set.
How this can be done with scalismo is explained in detail in the
quickstart guide.

Note, the quickstart guide assumes that the surfaces have already been
aligned. If you have not done this already, you would need to also do
this step prior to the registration. The general procedure is
explained in this tutorial video
https://www.futurelearn.com/courses/statistical-shape-modelling/3/steps/250347.

Best regards,

Marcel

> https://groups.google.com/d/msgid/scalismo/1bf004d9-4c95-480a-b306-a7a2ee363c45%40googlegroups.com.

[Rob Gathercole]

Hey Marcel, 

Thanks for the reply, really appreciate your guidance.I've been having difficulties accessing the futurelearn website but thankfully have access to the documents you refer to in Scalismo. Unfortunately i'm unable to bring my meshes (not faces this time) into alignment using the code in the 'building shape model' tutorial. With other software (meshmixer) i have converted all meshes to the same number of points but I get meshes in all different orientations. I'm assuming this is because they aren't aligned or is it because the code in the building shape models tutorial is specifically tailored to faces (e.g. the landmarks refer to key points on the face?)? 

I have had success aligning my meshes through the iterative method described in 'rigid alignment', and so have attempted to merge the two codes (i.e. using the rigid alignment script to bring the meshes into alignment and then building a shape model with the code from the deformation fields and onwards). Unfortunately this leads to a model bearing no resemblance to the imported meshes (See attached). I'm wondering if this is because the meshes are aligned but not corresponding? 

Is there anything you'd advise to help me troubleshoot the model?

Cheers,

Rob

[Marcel Luethi]

Dear Rob,

Your guess is right: Having the same number of points is not sufficient but the points have to be in correspondence. This means that the i-th point in all the meshes should refer to the same anatomical point on your surface. Correspondence allows you to pick a point (e.g. the one corresponding to the tip of the nose in a face) in the reference surface and find the same point automatically on all the surface. This is what ultimately enables the statistical analysis. 

A software like mesh mixer cannot do that. You need to perform a registration to find for every point in the reference the corresponding point in the other meshes. The relevant concepts are explained in week 6 of our course:

http://shapemodelling.cs.unibas.ch/StatisticalShapeModelling_2017.html (The videos/articles should be freely accessible).

There is also readily available code for registration, using the ICP method. You can run all this code directly from Scalismolab.

Best regards,

Marcel

To unsubscribe from this group and stop receiving emails from it, send an email to scalismo+unsubscribe@googlegroups.com.

To post to this group, send email to scal...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/scalismo/b5553deb-56da-456b-9e19-ebf1ac6f93c9%40googlegroups.com.

[Rob Gathercole]

Hey Marcel,

Thanks very much for your assistance so far. I've spent the weekend attempting to get the model up and running, and was actually successful! However, the model didn't perform as well as anticipated and looking back at some of the meshes I think my registration needs improving as the quality of the meshes from pre-to-post registration really decreases. I've attached a couple of images (see below) so that you can see an example of a mesh before and after. I've added the code i'm using  for registration below too. I've tried varying iterations, number of points, random vs. uniform mesh sampler, as well as a couple of different registration methods. No luck. Is there a way to register each mesh and not lose quality??

Thanks again for the help so far. I'm hoping i'm really close to getting the hang of things?!

Cheers,

Rob

// load deformation model // use code above to produce deformation model if not already produced

val shapeModel = StatismoIO.readStatismoMeshModel(new File("datasets/Ft/Def_mod/Model_Unldd_aligned.h5")).get

show(shapeModel, "Model")

// load reference mesh to enable goodness of fit assessment

val referenceMesh = MeshIO.readMesh(new File("datasets/Ft/Unldd_aligned/0.CP.stl")).get

show(referenceMesh, "referenceMesh")

// load target mesh

val targetMesh = MeshIO.readMesh(new File("datasets/Ft/Unldd_aligned/2.EB.stl")).get

show(targetMesh, "targetMesh")

val evaluationSampler = UniformMeshSampler3D(

    referenceMesh,

    numberOfPoints = 500,

    seed = 42

)

val regConfig = RegistrationConfiguration(

  optimizer = GradientDescentOptimizer(numIterations = 50, stepLength = 0.1),

  metric = MeanSquaresMetric(evaluationSampler),

  transformationSpace = GaussianProcessTransformationSpace(lowRankGP),

  regularizationWeight = 1e-8,

  regularizer = L2Regularizer

)

val fixedImage = Mesh.meshToDistanceImage(referenceMesh)

val movingImage = Mesh.meshToDistanceImage(targetMesh)

val regIterator = Registration.iterations(regConfig)(fixedImage, movingImage)

val resultIterator = for ((it, itnum) <- regIterator.zipWithIndex) yield {

  println(s"object value in iteration $itnum is ${it.optimizerState.value}")

  setCoefficientsOf("Model", it.registrationResult.parameters)

  it.registrationResult

}

val registrationTransform = resultIterator.toSeq.last

val projection = (pt : Point[_3D]) => {

  val targetPt = targetMesh.findClosestPoint(pt).point

  targetPt

}

val finalTransform = registrationTransform.andThen(projection)

val regResult = referenceMesh.transform(finalTransform)

show(regResult, "projection")

pre

post

[Marcel Luethi]

Hi Rob,

Congratulations, it looks already much better.

The artifacts, which you observe, are due to insufficiently accurate registration. There could be many reasons for that but looking at your code I think that one of the main problems is that you are using the GradientDescentOptimizer. 

For surface registration, the LBFGS Optimizer usually leads to much better results.

You can write:

val optimizer = LBFGSOptimizer(numIterations = 100)

For this to work well, you should also use more points in your evaluation sampler (we usually choose between 2000 - 5000).

This leads to more accurate computation of the distance metric and the gradients.

I am curious to hear if this helps in your application or if we need more tuning. 

Best regards,

Marcel

To unsubscribe from this group and stop receiving emails from it, send an email to scalismo+unsubscribe@googlegroups.com.

To post to this group, send email to scal...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/scalismo/a0dc156d-4e88-439d-90c3-b7915f816eaf%40googlegroups.com.