Problem using periodic boundary condition in smuthi

[Doan Le]

Dear all, 

I tried to run example 10 in the smuthi tutorial (link below). However there was an error that stopped the program (as shown in the figure). I am wondering if you've ever run into the same problem. I hope to have your help. Thank you.

example 10: https://gitlab.com/AmosEgel/smuthi/-/blob/master/examples/tutorials/10_periodic_near_field/perforated_glass_sheet.py 

Best regards,

[Amos Egel]

Dear Doan,

this looks like the cython extensions could not be loaded.

did the error occur on a system where the latest Smuthi commit had
been installed?

If not, please pull the latest commit, install Smuthi from scratch and
try again.

If the error still occurs, please post the full output to this thread.

Sorry for the inconvenience and thanks for your patience.

Amos

> --
> Smuthi project repository: https://gitlab.com/AmosEgel/smuthi
> Online documentation https://smuthi.readthedocs.io/en/latest/
> ---
> You received this message because you are subscribed to the Google Groups "Smuthi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to smuthi+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/smuthi/4035a995-770e-475a-8974-32f397b99f60n%40googlegroups.com.

[Doan Le]

Dear Amos,

Sorry for the late reply,

I have tried to reinstall both on my personal computer and server but it still didn't work. 

- When using pip to install: python -m pip install smuthi , the package didn't contain the 'periodicboundaries' folder. 

- When trying to clone from gitlab: there is problem importing Cython file (as shown in attach)

I hope that you can help.

Best,

Doan

Vào lúc 18:13:51 UTC+2 ngày Thứ Ba, 13 tháng 9, 2022, amos...@gmail.com đã viết:

[Parker Wray]

Hi Doan,

It is interesting that the periodic folder does not get added when installing smuthi. Thank you for bringing this to our attention. I will need to look into this when I have more time.

For now, I'd like to propose two options

Option 1:

Install smuthi based on the standard pip method (not cloning). Then, grab the periodicboundaries folder from gitlab and manually insert it into your installed local version of smuthi.

Option 2:

The automated install of SMUTHI performs the following steps:

1) install smuthi into specified default location. (This is usually a default location managed by python or conda).

2) run a script to compile c-level programs into machine code based on your computer-specific settings.

If you clone the repository from gitlab, then the automatic setup may fail if it attempts to compile c files and does not find them in the standard location or if it does compile the c files but stores them in a non-standard location.

The manual clone should not compile the c files to system-specific machine code. It just clones over what is on gitlab. So this should be the problem.

This is ok as the c files can be compiled manually. The error message is giving information on how to do that.

First, follow the instructions in compile_speedups_from_c.py

(If this does not work use the from_pyx.py file)

This will tell you how to manually compile the c files into machine code for your operating system.

This involves navigating to the cython_speedups folder in the command line. Then run compile_speedups_from_c.py

This will generate your system-specific machine code and save it into the newly generated "build folder".

Go into the build folder and grab the .o and .so files and paste them into the cython_speedups folder.

(Note: the generation of a build folder is just to make sure that compile_speedups_from_c.py does not overwrite pre-existing files.

 This is nice if you run on many different systems and want to save compiled versions of the c code for each system. You just change the name of the .o and .so files accordingly.

For you, do not worry about this. Just copy and paste.)

Once the files are compiled, you need to tell smuthi where the compiled files are. This is likely not a problem if "import smuthi" works for you. So I would first check if compiling the c code fixes your problem. If not,

Line 118 of the helper file is

import_wofz_possible_file_paths = ["cython_speedups", "smuthi.utility.cython.cython_speedups", "utility.cython.cython_speedups"]

This tells smuthi to look for the necessary compiled c files in the filepaths  = ["cython_speedups", "smuthi.utility.cython.cython_speedups", "utility.cython.cython_speedups"]

If you are running smuthi from a nonstandard location, (i.e., commands like "import smuthi' are being replaced by "import <my custom filepath>.smuthi"), then you need to specify this custom path in the list of file paths to check on line 118 as well.

Alternatively, you can run simulations from the smuthi directory then "import smuthi" will work and all subfiles can be properly found.

[Amos Egel]

Hey Parker,

thanks for your explanations!

I have one comment and one question ...

The installation from PyPi (i.e., "python -m pip install smuthi") doesn't create the periodicboundaries package because the package was added to the repository by Dominik after the release of version 1.2. So, until the next release, which will include that feature, the periodicboundaries package is available only by donwloading the current unreleased state of the repository from GitLab and then installing from source (i.e., "python -m pip install ." or "python -m pip install -e ." ).

My question is, in the error message that Doan sent contains the notion of a "non-standard directory" from which the code was run. So, what would be a standard directory? The root directory of the smuthi installation?

Cheers, Amos

To view this discussion on the web visit https://groups.google.com/d/msgid/smuthi/0c1a009a-f609-484c-9d69-a16d89b3e88cn%40googlegroups.com.

[Parker Wray]

Hi Amos,

I should have put an __init__.py file in the cython directory. I will do that now. Below is my explanation of the current implementation.

In the Ewald helper file of the periodic boundary module I added the code in the attached image. Wofz is a custom c function. Therefore, to use it, python needs to know where to find its .h file. The attached code searches 3 possible paths to find the .h file, which defines the custom c implementation of the wofz function. The hard coded possible search paths are "cython_speedups", "smuthi.utility.cython.cython_speedups", "utility.cython.cython_speedups". From my understanding, the rules for finding files using "get_cython_function_address" (a numba extension used for finding and reading .h files) should follow the same rules as the standard import used in python. If smuthi is in the search path (can be checked using sys.path), then  "smuthi.utility.cython.cython_speedups" is the folder path to the file. If, for example, you are running code and your working directory is inside smuthi, then "utility.cython.cython_speedups"  should catch the file. The "cython_speedups" should work if the cython folder is in sys.path (though, I forgot to add an __init__ file in the cython folder...).

I suspect having an __init__ file in the cython folder will add the cython subdirectory to path and that this will allow "get_cython_function_address"  to find the folder automatically. I did not find verification of this on Numba's website, so I hard coded the possible search paths just in case (then forgot to add the __init__).  For now, I would like to suggest we add a blank __init__ in the cython folder and keep the hard coded paths as well. The option "cython_speedups" assumes the subdirectory is in the search path already. So the hard coded options are then just redundant if the __init__ folder works as expected for "get_cython_function_address". I can then go back and check if things still work when the hard coded paths are remove. If so, I can push a new change.

Hopefully this helps.

Sincerely,

Parker

[Parker Wray]

Hi Doan,

We have made changes to SMUTHI that I hope should solve your issue. Can you please try to download it in the standard way and let us know if this works for you?

Sincerely,

Parker

On Wednesday, November 9, 2022 at 2:52:53 AM UTC-8 ledoa...@gmail.com wrote:

[Amos Egel]

The changes are not yet released on PyPi, so you'll need to download the latest commit from GitLab and install manually.

To view this discussion on the web visit https://groups.google.com/d/msgid/smuthi/2d239a6d-f168-4dc7-8c2e-d81f880c704fn%40googlegroups.com.

[Doan Le]

Thank Parker and Amos for the replies,

As Amos said, the periodic folder still can't be installed from PyPi. And I still have the same problem when cloning it from Gitlab. 

Hope it will be solved soon,

Doan 

Error message: "Cython acceleration could not be loaded. Falling back on Python equivalents...." and the following message is the same as the attach in the previous email. 

Vào lúc 20:44:01 UTC+1 ngày Thứ Tư, 16 tháng 11, 2022, amos...@gmail.com đã viết:

[amos...@gmail.com]

This thread was continued in private communication. According to Doan, the problem is now solved with the release of Smuthi version 2.