BFM 2017/2019 segmentation mask and parametric-face-generator questions
Arish Alreja
Hello everyone and a happy 2021
I'm a graduate student in computational neuroscience working on better understanding how the brain processes face related information. I want to first thank the maintainers of the BFM ecosystem of tools and documentation that you have assembled for other researchers to use. My familiarization with the BFM has been greatly accelerated by the resources available. Unfamiliarity with Scala/Java made me a little apprehensive when i started, but the resources were great and I'm hoping to use the BFM in my research in a very real way very soon.
After reading the papers, and exploring the resources, there are a few issues I was unable to resolve. Any advice or clarification would be appreciated. Apologies if these are answered elsewhere or naïve questions, I did my best to scour the resources before asking, but I'm also new to Computer Vision.
- Is the
parametric-face-generator application supposed to support the face only
models (i.e. model2017-1_face12_nomouth.h5, model2019_face12)?
- I hit exceptions when I try to use these files.
- I have no issues with the other model files for the 2017 or 2019 versions of the model.
- If it is not supported, that would be great to confirm. So far, I haven't found any resources that suggest that.
- If it is supported, I'd love to know if there is something specific I need to do to make this work? Or if I have run into a bug, is there anything I can do to help resolve it (exception traces, config files)?
- I ran into the same behavior on both a windows and ubuntu machine.
- Is there any support for
segmentation masks with the BFM 2017 or BFM 2019, like there was in BFM
09?
- I think the answer is no.
- I initially thought the region maps included with the face generator were examples of customizable masks, but I don't think this is the case.
- I was able to find 'level-mask-l7' and 'semantic-mask-l7' in one of the downloadable resources. They have the appearance of segmentation style masks (albeit a bit different in appearance than the one included with BFM 2009), though they are not officially discussed anywhere as serving the purpose of a mask.
- I felt I could use those masks, but realize I can't do a blended reconstruction like BFM 2009, because I don't know how to find or generate the segMM, segMB matrices that were present in the BFM 2009 model, for the 2017 or 2019 model.
- If segmentation masks are not possible or intended for 2017/2019, that would be great to confirm?
Regards,
Arish Alreja
egger.b...@gmail.com
- The reason why it might throw an error would most likely be that the face12 model does not have all the landmarks available: in the landmark-tags field of the config file, you can try to remove all landmarks at the ears since those don't exist in the face mask.
- if that is not the cause for the error you get it is easiest to send us the error message you receive when starting the tool on the console
- we however include segmentation mask that can be rendered in the parametric image generator (see Rendering Face Regions in the readme) https://github.com/unibas-gravis/parametric-face-image-generator if you need one of those masks for a different model (https://github.com/unibas-gravis/parametric-face-image-generator/tree/master/data/regions) we can provide them in case they are missing.
- and yes, you can also design your own mask by exchanging the png file