Data type error with "mask.fromfile" process in e2proc3d

[JoelM]

Hi,
I am using the "mask.fromfile" process in e2proc3d. When I issue the command, I receive an error message that mentions an inability to convert to a string. Am I perhaps passing the "filename" param incorrectly? Thanks in advance for the help/suggestions...
Joel

# Command 1: Create binary mask (no error)
e2proc3d.py OriginalMap.mrc mask.mrc --process=threshold.binary:value=1.11

# Command 2: Low pass filter the binary mask (no error)
e2proc3d.py mask.mrc mask_lp.mrc --process=filter.lowpass.gauss:apix=4.1:cutoff_freq=30

# Command 3: Apply the binary mask to the original map (gives error)
e2proc3d.py OriginalMap.mrc MaskedMap.mrc --process=mask.fromfile:filename=mask_lp.mrc

# Error output from command 3
Traceback (most recent call last):
  File "/Applications/EMAN2//bin/e2proc3d.py", line 503, in <module>
    main()
  File "/Applications/EMAN2//bin/e2proc3d.py", line 308, in main
    data.process_inplace(filtername, param_dict)
RuntimeError: TypeException at /Volumes/EMAN/co/eman2.daily/libEM/emobject.cpp:570: error with 'EMDATA': 'Cannot convert to string from this data type' caught

# Line 308 of e2proc3d.py
data.process_inplace(filtername, param_dict)

[Steven Ludtke]

Hi Joel. Technically processors aren't supposed to take filenames as arguments. Someone broke the conventions. The correct way to do this operation at present is to use the  --multfile=MULTFILE  option.

----------------------------------------------------------------------------
Steven Ludtke, Ph.D.
Professor, Dept. of Biochemistry and Mol. Biol.      Those who do

Co-Director National Center For Macromolecular Imaging            ARE

Baylor College of Medicine                                     The converse
slu...@bcm.edu  -or-  ste...@alumni.caltech.edu               also applies
http://ncmi.bcm.edu/~stevel

--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2

[Steven Ludtke]

That came out poorly, ie

e2proc3d.py OriginalMap.mrc mask.mrc --process=threshold.binary:value=1.11 --process=filter.lowpass.gauss:apix=4.1:cutoff_freq=30

e2proc3d.py OriginalMap.mrc MaskedMap.mrc --multfile=mask.mrc

However, I'd point out that this isn't really an appropriate mask in most cases, as the lowpass filter will work its way in towards the map as well as out away from the mask. To accomplish something more appropriate, you would probably put a mask.addshells processor after the the binary threshold, but before the filter.

However, what you're doing here is pretty similar, though a bit more primitive, than what the mask.auto3d processor does. Try running: 

e2filtertool.py OriginalMap.mrc

Then creating a mask.auto3d instance. You can then adjust the parameters interactively to achieve the mask you like. This is easier if you add a slice or two to the 3-D isosurface view. ie - it's hard to see the mask with an isosurface, but it's easy to see in a slice.

See:

http://blake.bcm.edu/emanwiki/EMAN2/Programs/e2filtertool

http://blake.bcm.edu/emanwiki/EMAN2/FAQ/AutoMask

----------------------------------------------------------------------------
Steven Ludtke, Ph.D.
Professor, Dept. of Biochemistry and Mol. Biol.      Those who do

Co-Director National Center For Macromolecular Imaging            ARE

Baylor College of Medicine                                     The converse
slu...@bcm.edu  -or-  ste...@alumni.caltech.edu               also applies
http://ncmi.bcm.edu/~stevel

[je...@crystal.harvard.edu]

Hi Steve,

The same crash also happens when you run e2refine_easy.py with the option

--automask3d2=mask.fromfile:filename=custom_mask.hdf:ismaskset=0.0

My work around for now was to replace in e2proc3.py this lines of code:

#Parse the options to convert the image file name to EMData object(for both plain image file and bdb file)

for key in param_dict.keys():

if str(param_dict[key]).find('bdb:')!=-1 or not str(param_dict[key]).isdigit():

try:

param_dict[key] = EMData(param_dict[key])

except:

pass

with:

#Parse the options to convert the image file name to EMData object(for both plain image file and bdb file)

for key in param_dict.keys():

if str(param_dict[key]).find('bdb:')!=-1 or not str(param_dict[key]).isdigit():

try:

if (filtername=="mask.fromfile"):

param_dict[key] = param_dict[key]

else:

param_dict[key] = EMData(param_dict[key])

except:

pass

Just to let you know and if someone runs into the same problem.

All the best, Simon

[Steve Ludtke]

Hi Simon. We did actually implement a solution to the problem you originally reported in 2012. It should be possible to specify any processor option in e2proc3d.py as a filename, and this should extend to this option in e2refine_easy.  What is your full e2version.py output?

----------------------------------------------------------------------------
Steven Ludtke, Ph.D.
Professor, Dept. of Biochemistry and Mol. Biol.                Those who do

Co-Director National Center For Macromolecular Imaging            ARE

Baylor College of Medicine                                     The converse
slu...@bcm.edu  -or-  ste...@alumni.caltech.edu               also applies
http://ncmi.bcm.edu/~stevel

-- 

-- 
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2

--- 

You received this message because you are subscribed to the Google Groups "EMAN2" group.

To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[je...@crystal.harvard.edu]

e2version.py gives:

EMAN 2.12 (CVS 2015/10/19 09:00:04)

Your EMAN2 is running on:  Linux-3.10.0-229.11.1.el7.x86_64-x86_64-with-redhat-7.1-Nitrogen 3.10.0-229.11.1.el7.x86_64 x86_64

Your Python version is:  2.7.3

With this version, this command fails:

e2proc3d.py input.hdf output.hdf --process mask.fromfile:filename=custom_mask.hdf:ismaskset=0.0

Traceback (most recent call last):

  File "/programs/x86_64-linux/eman2/2.12/bin/e2proc3d.py", line 770, in <module>

    main()

  File "/programs/x86_64-linux/eman2/2.12/bin/e2proc3d.py", line 464, in main

    else : data.process_inplace(filtername, param_dict)

RuntimeError: std::exception

The problem is that 

param_dict[key] = EMData(param_dict[key])

doesn't yield the filename as string, but rather an address of the python object.

All the best, Simon

[Steven Ludtke]

Ahh, sorry, my mistake.

By design, EMAN2 processors were never supposed to take filenames as arguments. If another image were required as input, it was supposed to be passed in as an image object, not as a filename. The reasoning behind this is that some of the parallelism mechanisms do not provide access to the filesystem, so if a filename were provided, the file might not be accessible in all situations. In earlier versions of e2proc3d, specified filenames would not be automatically read and converted into images, but this capability was later added so processors requiring additional image inputs could be used with e2proc3d. This change fixed those, but broke mask.fromfile.

This will require a patch :^(

[je...@crystal.harvard.edu]

Thanks, for the explanation Steve.

[Steven Ludtke]

Ok. I just committed a change which will appear in today's snapshot. mask.fromfile now can take either a filename or an image as input, and also properly supports multi-level mask files. While it won't help with the old release, moving forward no patching the python should be required to use this with e2refine_easy.