numpy2em problem

[Tapu Shaikh]

I encountered some strange behavior with EMNumPy.numpy2em() under Python 3.  If I run the following, I get a blank array under Python 3 but not Python 2:

import numpy as np aa = np.arange(9).reshape(3, 3) img = EMNumPy.numpy2em(aa) img.get_2dview()

If I cast the array as float32, things are OK again:

aa = np.arange(9).reshape(3, 3).astype(np.float32) img = EMNumPy.numpy2em(aa) img.get_2dview()

However, when I compute an FFT and inverse FFT, the output is scrambled relative to the starting array, and only in Python 3:

aa = np.arange(9).reshape(3, 3).astype(np.float32) img = EMNumPy.numpy2em(aa) imgarray = img.get_2dview() imfft = np.fft.fft2(imgarray) real_array = np.fft.ifft2(imfft) eman_real = EMNumPy.numpy2em(real_array.real) eman_real.get_2dview()

Note 1: Without the ".real" in numpy2em, I get a blank array again, only in Python 3. 

Note 2: This was the original scenario when I encountered this discrepancy.

Now, if in numpy2em I cast as float32 explicitly, things work again in Python 3:

aa = np.arange(9).reshape(3, 3).astype(np.float32) img = EMNumPy.numpy2em(aa) imgarray = img.get_2dview() imfft = np.fft.fft2(imgarray) real_array = np.fft.ifft2(imfft) eman_real = EMNumPy.numpy2em( real_array.real.astype(np.float32) ) eman_real.get_2dview()

However, I then get a 90-degree rotation in Python 2 (!).

Markus has a function called sp_utilities.numpy2em_python() which works in both Python 2 & 3.  He'd have to explain why it works. 

Looking quickly, e2proc2d uses numpy2em() once.  Someone may want to check on it.

Unrelated: In programs/e2spt_average.py:56, someone used double quotes inside double quotes: if verbose: print(f"{i} {s}: {d["xform.align3d"].get_trans}")

Is it OK if I fix it?  One of my builds is failing because of it.

-Tapu

[Steve Ludtke]

Hi Tapu,

we did revamp the NumPy interface for Python 3, as the mechanism we were using previously was deprecated by numpy. We are not supporting the Python2 interface in any way any more. 

The Python 3 interface uses shared memory with the NumPy object, and as far as we can tell is properly handling object sharing so it doesn't crash under normal use-cases. This avoids the CPU and RAM overhead of copying all of the time. It does mean that only float32 NumPy arrays can be converted to/from EMData objects.

I'm not sure what led you to the get_2dview() interface. As far as I know, that set of interfaces (along with 3d and complex counterparts) was deprecated over a decade ago. I don't believe it is used anywhere in the system at present (I could be wrong). I think it may have been originally implemented before we got the numpy sharing figured out?  Anyway, I would not recommend using those functions at all as I have no idea if they work properly. Actually, my normal way to view an EMData object like this would be to use the NumPy conversion  :^)

As far as I am aware all of the Python3 NumPy functionality is working correctly. The only possible exception is performing FFTs in NumPy then trying to move the complex data to EMAN2. This may work properly, but we have never tested it. All FFTs are normally computed on the EMAN2 side.

import numpy as np

aa=np.arange(9).reshape(3,3).astype(np.float32)

img=from_numpy(aa)

aa = np.arange(9).reshape(3, 3).astype(np.float32)

img = from_numpy(aa)

imgf = img.do_fft()

imfft = to_numpy(imgf)

--
You received this message because you are subscribed to the Google Groups "EMAN2 and SPARX Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eman2-develope...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/eman2-developers/2685a165-477c-ae8f-bd7a-d312a2fc7ae9%40gmail.com.