Build Up BFM-2017

Xiang Li

unread,

Jun 13, 2019, 3:48:40 PM6/13/19

to scalismo-faces

Hi,

I don't know if this group is still functional, but it is worth to try.

I tired to construct 3D face model from 2D image using BFM-2017. The paper provides a link to Github to BFM pipeline code. But I also signed up on the BFM-2017 website and it sent me an email with BFM model to a software called statismo. I am so confused now. Which one I should use to do the detection? Or those two are just two softwares with the same function, and I can use either one?

Thanks for help. Looking forward to your replay.

Bernhard Egger

unread,

Jun 13, 2019, 4:27:40 PM6/13/19

to scalismo-faces

Hi Xiang Li,

the code to reconstruct a 3D face from a 2D image is provided with the basel face pipeline:

https://github.com/unibas-gravis/basel-face-pipeline

What you are looking for is the last step "Face Reconstruction from 2D Image" with this you can reproduce the results in our paper "Morphable Face Models - An Open Framework"

To reconstruct other than those images, you can adapt https://github.com/unibas-gravis/basel-face-pipeline/tree/master/src/main/scala/fitting/experiments or call the fitting script directly https://github.com/unibas-gravis/basel-face-pipeline/blob/master/src/main/scala/fitting/StandardFitScript.scala

In case you are not familiar with scala or our approach to 3D reconstruction we provide a nice tutorial here:

https://gravis.dmi.unibas.ch/PMM/lectures/fitting/

Best

Bernhard

Xiang Li

unread,

Jul 2, 2019, 7:36:10 PM7/2/19

to scalismo-faces

Thank you for your replay Bernhard,

I am facing a problem that it takes almost 3 days to do the step 4: Registration and it still going on now. Is that normal?

Here is some outputs from the running code:

Screen Shot 2019-07-02 at 19.33.02.png

Screen Shot 2019-07-02 at 19.28.24.png

Some of the outputs looks good. They are doing the iterations. But some of them have errors. Would that causes the long time running, since it has error and it keeps running repeatedly?

Thank you so much for your time and answering my questions.

Best,

Xiang Li

Screen Shot 2019-07-02 at 19.28.24.png

Xiang Li

unread,

Jul 9, 2019, 12:09:09 PM7/9/19

to scalismo-faces

Hi,

I am still waiting for this answer. My program is still running. It has been running about one week. Is that normal? What should I do to improve the running time? I am so desperate now. Looking forward to anyone reply.

Best,

Xiang Li

Bernhard Egger

unread,

Jul 9, 2019, 1:04:58 PM7/9/19

to scalismo-faces

Hi Xiang Li,

I'm not that familiar with that part of the pipeline. Can you check if it is producing any files in your output folder? This could give you an intuition if it is running and how long it might go on.

From your first question, I guess that you are actually just interested in the 3D reconstruction from 2D images: In that case you don't have to go through the whole pipleine to build a model. Just download the basel face model and run the face reconstruction directly "Face Reconstruction from 2D Image" on https://github.com/unibas-gravis/basel-face-pipeline

If you don't change the code it will crash when trying to fit the bu3d model (since this is the part you can skip), but this is when the fittings for the basel face model are already done

https://github.com/unibas-gravis/basel-face-pipeline/blob/master/src/main/scala/fitting/experiments/QualitativeLFW.scala

We also provide a tutorial to better understand the fitting:

http://gravis.dmi.unibas.ch/PMM/lectures/fitting/

Best

Bernhard

Xiang Li

unread,

Jul 10, 2019, 12:47:20 PM7/10/19

to scalismo-faces

Thank you Bernhard,

I just finished running step 4. It took me 9 days. Thank you for your help. However, I am wondering on step 5. It says you have to comment out some lines. Do you have some clue which lines I should comment out? I looked through the code I have no idea which lines I should comment out?

Best,

Xiang Li

Bernhard Egger

unread,

Jul 10, 2019, 1:00:09 PM7/10/19

to scalismo-faces

Can you try without commenting anything out first?
I assume it is referingto line 50 here:
https://github.com/unibas-gravis/basel-face-pipeline/blob/master/src/main/scala/modelbuilding/ModelBuilding.scala

The mask that is set by default is the one you want for the fittings

Message has been deleted

Xiang Li

unread,

Jul 16, 2019, 10:10:19 AM7/16/19

to scalismo-faces

I know I always ask silly question. But still thank you for you reply.

I think the module runs find now. However, it is not working appropriately, The attaches are the original image and the it's corresponding 3D face. But there is only nose that been shown in the image.

I assume maybe I can not use RecofnitionMulriPIE.scala:

https://github.com/unibas-gravis/basel-face-pipeline/blob/master/src/main/scala/fitting/experiments/RecognitionMultiPIE.scala

Instead I should have use QualitativeLFW.scala:

https://github.com/unibas-gravis/basel-face-pipeline/blob/master/src/main/scala/fitting/experiments/QualitativeLFW.scala

Or StandarFitScript.scala to reconstruct 3D face from 2D image right?

https://github.com/unibas-gravis/basel-face-pipeline/blob/master/src/main/scala/fitting/StandardFitScript.scala

Again, sorry for my silly question. Your work is so excellent, I really need to let it works.

Thank you for you patient and reply.

Best,

Xiang Li

fitter-best.png

target.png

Bernhard Egger

unread,

Jul 16, 2019, 10:15:20 AM7/16/19

to scalismo-faces

Hi Xing Li,

that output is really surprising - I have no idea what happened there. Did you provide landmarks with the target image?

You can use this tool to click landmarks:

https://github.com/unibas-gravis/landmarks-clicker

You can adopt either the multipie or lfw script - they both just call the StandardFitscript.

Alternatively you can follow our tutorial to get an introduction in how our pipeline works:

http://gravis.dmi.unibas.ch/PMM/lectures/fitting/

Best

Bernhard

Xiang Li

unread,

Jul 16, 2019, 10:19:36 AM7/16/19

to scalismo-faces

Hi Bernhard Egger,

I think my database has its own ibug68 landmarks. And I manually converted it to .tlms file which defined 7 points on face.

I think I need to review the introduction to learn how it works. Thank you for your reply. I will keep updating my result to you in case you need some feedback.

Thank you again.

Sincerely,

Xiang Li

Bernhard Egger

unread,

Jul 16, 2019, 10:22:11 AM7/16/19

to scalismo-faces

Hi Xiang Li,

could you share the tlms file? just to see if that went well?

You might also consider downsampling the image to something like 500*500 pixels or so - it will be much faster and some of the steps of the pipeline are most tested in that resolution (you will need to change the landmarks too).

Best

Bernhard

Xiang Li

unread,

Jul 16, 2019, 10:25:21 AM7/16/19

to scalismo-faces

Here is the tlms for the Harry Potter image:

outdoor_084_face0.tlms

Bernhard Egger

unread,

Jul 16, 2019, 10:32:20 AM7/16/19

to scalismo-faces

Hi Xiang Li,

left and right is mixed up :)

That way you fit the mask and only some vertices facing the image plane in the nose region are rendered.

Our conversion with landmarks is that they are seen from the face. So right eye would be your right eye, not the right one from the perspective of somebody looking at you.

I would still recommend to downsize the image and landmarks - it will be much faster that way.

Best

Bernhard

Xiang Li

unread,

Jul 16, 2019, 10:35:23 AM7/16/19

to scalismo-faces

Aha! Bang! Silly me!

Thank you Bernhard, that why I got wrong. I will run one time more to see the result. I will update my result to you.

Bernhard Egger

unread,

Jul 16, 2019, 10:37:44 AM7/16/19

to scalismo-faces

Just in case you run into issues with occlusions (like glasses). We also share a tutorial on that:

http://gravis.dmi.unibas.ch/PMM/lectures/segmentation/

:)

Best

Bernhard

Reply all

Reply to author

Forward