rsgislib utilising GPU

[Girish Kumar]

Hi all,

         We are processing a huge number of Sentinel 2 images(extracting,stacking and then applying band math for implementing some algorithms using rsgislib). But it takes more than 30 minutes to process a single image on a workstation(32 GB RAM,Dual Intel Xeon processors with windows 7 professional).rsgislib is setup on A VM with RHEL 7 with 18 GB RAM allocated to it. The server is high-end one, It has NVIDIA graphics card(NVIDIA Tesla). Is it possible to use the GPU for processing images so that the speed can be increased. Also is it possible to do parallel processing with rsgislib.

Thanks and Regards

Girish Kumar

[Daniel Clewley]

Hi Grish,

It isn’t possible to use the GPU within RSGISLib. The best way to run in parallel is to have different scenes running on different cores or machines, this is the way RSGISLib has been used on a HPC. Alternatively you can split a scene into tiles and run each tile on a separate core. There are versions of some algorithms within RSGISLib which will split a scene into tiles as part of the processing.

The operations you mention (extracting, stacking and band maths) are normally I/O rather than CPU limited so do you also have fast disks? If you run multiple process at once you also need to make sure the disks are fast enough to cope.

Thanks,

Dan

--
You received this message because you are subscribed to the Google Groups "RSGISLib Support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-suppo...@googlegroups.com.
To post to this group, send email to rsgislib...@googlegroups.com.
Visit this group at https://groups.google.com/group/rsgislib-support.
For more options, visit https://groups.google.com/d/optout.

[Girish Kumar]

Hi Dan,

            Thanks for the quick reply.  The system has about 16 cores. Is there any direct method in rsgislib to run different scenes on different cores in parallel, or any other python library to be used to achieve this?

Thanks and regards

Girish Kumar

With Warm Regards

T P Girish Kumar

On Mon, Mar 26, 2018 at 1:17 PM, Daniel Clewley <daniel....@gmail.com> wrote:

Hi Grish,

It isn’t possible to use the GPU within RSGISLib. The best way to run in parallel is to have different scenes running on different cores or machines, this is the way RSGISLib has been used on a HPC. Alternatively you can split a scene into tiles and run each tile on a separate core. There are versions of some algorithms within RSGISLib which will split a scene into tiles as part of the processing.

The operations you mention (extracting, stacking and band maths) are normally I/O rather than CPU limited so do you also have fast disks? If you run multiple process at once you also need to make sure the disks are fast enough to cope.

Thanks,

Dan

On 26 Mar 2018, at 05:49, Girish Kumar <giris...@gmail.com> wrote:

Hi all,

         We are processing a huge number of Sentinel 2 images(extracting,stacking and then applying band math for implementing some algorithms using rsgislib). But it takes more than 30 minutes to process a single image on a workstation(32 GB RAM,Dual Intel Xeon processors with windows 7 professional).rsgislib is setup on A VM with RHEL 7 with 18 GB RAM allocated to it. The server is high-end one, It has NVIDIA graphics card(NVIDIA Tesla). Is it possible to use the GPU for processing images so that the speed can be increased. Also is it possible to do parallel processing with rsgislib.

Thanks and Regards

Girish Kumar

--
You received this message because you are subscribed to the Google Groups "RSGISLib Support" group.

To unsubscribe from this group and stop receiving emails from it, send an email to rsgislib-support+unsubscribe@googlegroups.com.
To post to this group, send email to rsgislib-support@googlegroups.com.

[Daniel Clewley]

Hi Girish,

You can use multiprocessing in Python, I wrote an example here: https://spectraldifferences.wordpress.com/2013/11/21/multi-core-processing-with-rsgislib-using-tiles/

If you have a command line tool you are needing to run on multiple cores GNU parallel is very good.

Thanks,

Dan

[Girish Kumar]

Hi Dan,

       Thanks for the reply and the link. I will try it. Also I was going through the numba python libraries, which seems to be good for GPU processing.

​Thanks and Regards

T P Girish Kumar