CUDA/OpenCL for pySPH

[Lucas Sanders]

Hello,

I am trying to install pySPH with the ability to run on an Nvidia Tesla K80 GPU. Do I need to install both pyOpenCL and pyCUDA for this to work with the current version of pySPH?

When I installed pyCUDA, then pySPH, and ran the tests, 21 tests failed and many CUDA tests were skipped due to not having pyOpenCL installed. However when I installed pyOpenCL and reinstalled pySPH, 158 tests failed (including some that passed in the previous case). I have attached the failures of the test logs for both cases.

Additionally, I tried installing both pyOpenCL and pyCUDA with pySPH 1.0a6 and all tests passed. Is CUDA supported in the current github version of pySPH?

Thanks,

Lucas

[Prabhu Ramachandran]

Hi Lucas,

I am sorry for the slow response.  I (or any others) have not paid much attention to the OpenCL/CUDA support in a while and it looks like there may be changes in either pytools or pyopencl that has broken things.  I was using older versions and it used to work as of 6-9 months ago. Compyle is reasonably tested and works but the pysph tests were broken and some of the OpenCL tests were failing.  I am currently traveling without access to any of our GPU machines but will try to fix any issues in the next few weeks.

Looking at your failures, they are too extensive for it to be of any use.  With the version on github, CUDA used to work as would OpenCL, however, the performance requires a lot more attention.  I will try to take a quick look at the errors this weekend, it might be a simple issue that is tripping up most of the tests.

Regards,
Prabhu

--
You received this message because you are subscribed to the Google Groups "pysph-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pysph-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pysph-users/2c9d4cc5-9f96-4a76-a95e-d5973f417c06n%40googlegroups.com.

[Prabhu Ramachandran]

On 8/26/21 21:45, Prabhu Ramachandran wrote:

I am sorry for the slow response.  I (or any others) have not paid much attention to the OpenCL/CUDA support in a while and it looks like there may be changes in either pytools or pyopencl that has broken things.  I was using older versions and it used to work as of 6-9 months ago. Compyle is reasonably tested and works but the pysph tests were broken and some of the OpenCL tests were failing.  I am currently traveling without access to any of our GPU machines but will try to fix any issues in the next few weeks.

Looking at your failures, they are too extensive for it to be of any use.  With the version on github, CUDA used to work as would OpenCL, however, the performance requires a lot more attention.  I will try to take a quick look at the errors this weekend, it might be a simple issue that is tripping up most of the tests.

I have fixed the issues and all the OpenCL and CUDA tests should pass now.  You will need the latest github version of compyle and pysph.  I have also re-enabled the opencl tests on github so hopefully we should catch these sooner.

cheers,
Prabhu

[Lucas Sanders]

Hi Prabhu,

Thank you for your help!

Regards,

Lucas

[Stephan]

Dear Prabhu,

In my previous comment on the 16th of December I attached a failures log. Since then I tried several things and currently I've reached the same situation as Lucas mentioned in his first comment. I have the exact same failures relating to pyopencl not being found as shown in the attached CUDA_only.txt file. I installed CUDA Toolkit on my system and I've tested CUDA, as well as pycuda and cupy and all of them seem to be working perfectly fine separately. To me it's not really clear if pyopencl is required or not, or if it is possible to run the code with CUDA only.

I will remove my previous comment since it is obsolete now, but I would really appreciate help regarding this issue that Lucas posted here earlier.

Best,

Stephan

Op maandag 30 augustus 2021 om 08:47:17 UTC+2 schreef Prabhu Ramachandran:

[Prabhu Ramachandran]

Hi Stephan,

I took a quick look at your log files. I just ran the compyle and pysph tests
on a machine with an nVidia GeForce GTX 1070 Ti and they all pass for me. So I
am not sure what is going on. I just pushed a fix to compyle to address the
np.bool deprecation warnings.

As regards CUDA and OpenCL, on my simple tests I tend to run pysph only using
OpenCL since I have found it to be faster than the CUDA backend.

Could you please let me know the exact steps you have taken?

cheers,
Prabhu


On 12/17/21 02:15, Stephan wrote:
> Dear Prabhu and Lucas,
>
> I have been running the code on a CPU for a while now but as simulations get
> bigger it became very time consuming so I wanted to use my GPU. Unfortunately, I
> have some failures reported when I run the pysph test. I have installed cupy,
> PyCUDA and PyOpenCL. I have been trying several things but I can't figure out
> where it goes wrong. Hopefully, one of you can help me out. I have attached the
> captured log call including the failures reported.
>
> Kind regards,
>
> Stephan
>
> Op maandag 30 augustus 2021 om 17:09:26 UTC+2 schreef lucas.sa...@gmail.com:

[Prabhu Ramachandran]

On 1/11/22 22:07, Stephan wrote:
> Dear Prabhu,
>
> In my previous comment on the 16th of December I attached a failures log. Since
> then I tried several things and currently I've reached the same situation as
> Lucas mentioned in his first comment. I have the exact same failures relating to
> pyopencl not being found as shown in the attached CUDA_only.txt file. I
> installed CUDA Toolkit on my system and I've tested CUDA, as well as pycuda and
> cupy and all of them seem to be working perfectly fine separately. To me it's
> not really clear if pyopencl is required or not, or if it is possible to run the
> code with CUDA only.
>
> I will remove my previous comment since it is obsolete now, but I would really
> appreciate help regarding this issue that Lucas posted here earlier.

I suspect we will require both pyopencl and pycuda. I am not sure we have
tested the case where someone has only pycuda and no pyopencl installed. Is
this is a major problem that we need to test for? As I said in my previous
email, opencl support is generally better than CUDA as far as pysph goes.

cheers,
Prabhu

[Stephan]

Dear Prabhu,

Thank you for your reply. Since my last post I have been working on the situation as well. After a while I finally got the code working with OpenCL, I'm not even going to try CUDA anymore. There were several issues I had to deal with because of my hardware configurations.

I am using a relatively old GPU (GeForce GTX 680) which only has compute capability 3.0. CUDA SDK 9.0 and 10.0 are the only ones supporting compute capability 3.0. Then I figured out that CuPy is being used in the code and CuPy only works with CUDA Toolkit 10.2 or higher. That means I was forced to use CUDA Toolkit 10.2 due to the given constraints. Next problem was that CUDA Toolkit 10.2 wouldn't work on Ubuntu 20.04 so I had to downgrade my OS to Ubuntu 18.04. After fixing all of that I managed to run tests with CuPy, PyCUDA and PyOpenCL which all worked well. After that I installed PySPH and it worked fine.

After a while I had some updates and suddenly PySPH stopped working again. I traced down all possible updates that trigged this issue and it turned out that when I installed an older version of PyOpenCL it was working again. Therefore, I now stick with PyOpenCL 2021.2.9. I'm not sure if there is a newer version that does work, but I tried several versions (from 2021.2.10 up to 2021.2.13) and all of them made PySPH unusable.

Anyway, for me there is no further need to test with CUDA since now it is working for me with PyOpenCL. Nevertheless, there are still some issues which I'm not getting resolved as they should. I will put them in a new thread since these are not related specifically to the header of this thread.

Thank you again, kind regards,

Stephan

Op donderdag 24 februari 2022 om 18:58:36 UTC+1 schreef Prabhu Ramachandran:

[Stephan]

Dear Prabhu,

The issue I was referring to in the last line can be found in the thread starting with: "TypeError: __init__(): incompatible constructor arguments . . . "

Since the creation of that thread I kind of tracked down the issue and made myself a work around, but I think this is not the correct and expected behavior. I will explain more details in the TypeError thread.

Best,

Stephan

Op woensdag 2 maart 2022 om 12:30:14 UTC+1 schreef Stephan: