Origin of Model

142 views

morphable-face-models---an-open-framework

Skip to first unread message

Bernhard Egger

unread,

May 31, 2018, 2:30:56 PM5/31/18

to scalismo-faces

The origin [0, 0, 0] of the model lies in the center of the head. This leads to not so nice effects when rotating with scalismo-faces (the face moves away from the center of the image).

With a full head this makes sense but for my applications (mainly the parametric-face-image-generator) with the bfm-mask, a point between the checkbone makes more sense. I wrote a code snippet that does this transformation - it might be useful for others too :) Basically the origin is moved to a point halfway to the nosetip

package scalismo.statisticalmodel

import java.io.File

import breeze.linalg.DenseVector
import scalismo.common.UnstructuredPointsDomain
import scalismo.faces.color.RGB
import scalismo.faces.io.MoMoIO
import scalismo.faces.momo.{MoMo, PancakeDLRGP}
import scalismo.geometry.{Point, Vector, _3D}


object ModelCenterPointChanger extends App{

  scalismo.initialize()

  val momo = MoMoIO.read(new File("model2017-1_bfm_nomouth.h5")).get

  val noseTip = momo.landmarkPointId("center.nose.tip").get
  val noseTipCoordinates = momo.referenceMesh.pointSet.point(noseTip)
  println(noseTip)
  println(noseTipCoordinates)

  val newReference = momo.referenceMesh.copy(pointSet = momo.referenceMesh.pointSet.transform(p => p - noseTipCoordinates.toVector / 2.0))
  val newLandmarks = momo.landmarks.map(l => (l._1, l._2.copy(point = l._2.point - noseTipCoordinates.toVector / 2.0 )))
  val newShapemean = momo.neutralModel.shape.mean.data.map(p => (p - noseTipCoordinates.toVector / 2.0).toArray).toArray
  val newShape = PancakeDLRGP[_3D, UnstructuredPointsDomain[_3D], Point[_3D]](new DiscreteLowRankGaussianProcess[_3D, UnstructuredPointsDomain[_3D], Point[_3D]](newReference.pointSet, DenseVector(newShapemean.flatten), momo.neutralModel.shape.variance, momo.neutralModel.shape.basisMatrix))
  val newColor = PancakeDLRGP[_3D, UnstructuredPointsDomain[_3D], RGB](new DiscreteLowRankGaussianProcess[_3D, UnstructuredPointsDomain[_3D], RGB](newReference.pointSet, momo.neutralModel.color.meanVector, momo.neutralModel.color.variance, momo.neutralModel.color.basisMatrix))
  val newExpression = PancakeDLRGP[_3D, UnstructuredPointsDomain[_3D], Vector[_3D]](new DiscreteLowRankGaussianProcess[_3D, UnstructuredPointsDomain[_3D], Vector[_3D]](newReference.pointSet, momo.expressionModel.get.expression.meanVector, momo.expressionModel.get.expression.variance, momo.expressionModel.get.expression.basisMatrix))
  val newMomo = MoMo(newReference, newShape, newColor, newExpression, newLandmarks)

  val newNoseTip = newMomo.landmarkPointId("center.nose.tip").get
  val newNoseTipCoordinates = newMomo.referenceMesh.pointSet.point(newNoseTip)
  println(newNoseTip)
  println(newNoseTipCoordinates)

  MoMoIO.write(newMomo, new File("model2017-1_bfm_nomouth_rotationCenterHalfwayToNose.h5"))

}

Bernhard Egger

unread,

Jun 5, 2018, 6:23:31 PM6/5/18

to scalismo-faces

There is actually a problem with the landmarks.

Replacing:

val newLandmarks = momo.landmarks.map(l => (l._1, l._2.copy(point = l._2.point - noseTipCoordinates.toVector / 2.0 )))

by

val newLandmarks = momo.landmarks.map(l => {
  val newpoint = newReference.pointSet.findClosestPoint(l._2.point - noseTipCoordinates.toVector / 2.0 ).point
  val pointfloat = newpoint.toArray.map(_.toFloat)
  (l._1, l._2.copy(point =  Point3D(pointfloat(0), pointfloat(1), pointfloat(2))))})