Pyresample Survey Results and Roadmap to 2.0

[David Hoese]

Hi everyone,

About a month and a half ago I sent out a link to the Pyresample User Survey. This was to gather information from our users on what they use pyresample for, what they think is good about pyresample, and what is missing. I've looked through the results and here is my summary:

• 22 responses
• Everyone uses pyresample for work and a few also use it in their hobby
• ...and most of that work is satellite data based
• People use pyresample because of the quality of results and the performance with which it produces those results
• Most people feel they have an intermediate (3+ out of 5) or better understanding of projections and coordinates in those projections, but only 2 people said they were experts on the topic (5 out of 5)
• Ideas for missing features in pyresample ranged everywhere from needing more documentation or better understanding of certain concepts (radius of influence, etc) to new resampling algorithms including temporal resampling support. There were also multiple responses related to providing defaults for some resampling parameters.
• Documentation was considered to be lacking the most and needing improvements including example figures. The second most mentioned component that needs work is the interfaces for accessing features.
• There were multiple responses around the idea of being able to access the intermediate index results of resampling.

I would say that a majority of the responses were inline with what I expected...except for the few that said the documentation is great (I personally agree with the idea that the docs need work).

The core pyresample devs met virtually a couple times to discuss this survey and to describe a path forward for a better Pyresample. We've summarized these ideas in a new "Roadmap" page on the documentation:

https://pyresample.readthedocs.io/en/latest/roadmap.html

I plan on working towards Pyresample 2.0 over the next 6-12 months. If you have any feedback on this roadmap feel free to respond to this mailing list post or talk to me in the "#pyresample" channel on the Pytroll slack (http://pytroll.github.io/#getting-in-touch).

Thanks to everyone who completed the survey and provided feedback.

Dave

[Thomas Lavergne]

Hello Dave,

Good to read that the survey was useful, and that you could outline a roadmap towards 2.0. The roadmap looks good to me (it captures my use-cases, and the proposed interfaces look great). I'll help testing the new interfaces while they evolve.

Concerning the Index() idea:

I note that one thing that is very neat with get_neighbour_info() is that kdtree.resample_*() actually calls get_neighbour_info() at its core: we can see in the resample_*() code how to use its ins/outs, and we are sure that they use the same (optimized & maintained) code base (e.g. handle masked values the same, etc...).

In the Roadmap you describe the Index() as a separate interface and "something separate from resamplers" and from the Cache.

I am thus wondering: do you foresee that (some of) the resamplers will be built upon (making use of) the Index interfaces? By the same token, could the Index implementation also benefit from Cache objects? Then we keep everything close together as in pyresample 1?

This was maybe what you had in mind anyways. If not, consider this a suggestion/request for the roadmap.

I am looking forward to this journey towards 2.0, and am so glad we have you in the driver's seat.

Cheers,

Thomas

[David Hoese]

Hi Thomas,

I don't know if we'll know all of the answers to this until we actually
implement all of it. I do think some resamplers will use Index objects
internally where possible and reducing the amount of duplicated code
between Resamplers and Index objects will be important. The main purpose
of the Index objects is to make it so people who actually just want
indexes don't have to use a Resampler. They can also be used in Xarray
if we set them up correctly to allow quick slicing of Xarray
DataArrays/Datasets.

I had assumed that people who wanted to use Index objects would not
necessarily need to cache the results. This of course could be possible,
I just didn't plan on it and I'm not exactly sure where the lines would
be drawn in the design. For example, do Index objects use Cache objects
that are passed by the user or does the user use a Cache object directly
and pass the results of the Indexing. We had talked at one point about
having a split resampling interface where the user always has to get
indexes and then calls a resampler method to use the indexes. We ended
up not wanting that as it may not apply to all resampling
algorithms/techniques. At the end of the day this is software so I see
no reason that we can't make the Index objects do everything we need
them to.

Thanks for the feedback.

Dave

> * 22 responses
> * Everyone uses pyresample for work and a few also use it in their
> hobby
> * ...and most of that work is satellite data based
> * People use pyresample because of the quality of results and the

> performance with which it produces those results

> * Most people feel they have an intermediate (3+ out of 5) or

> better understanding of projections and coordinates in those
> projections, but only 2 people said they were experts on the
> topic (5 out of 5)

> * Ideas for missing features in pyresample ranged everywhere from

> needing more documentation or better understanding of certain
> concepts (radius of influence, etc) to new resampling algorithms
> including temporal resampling support. There were also multiple
> responses related to providing defaults for some resampling
> parameters.

> * Documentation was considered to be lacking the most and needing

> improvements including example figures. The second most
> mentioned component that needs work is the interfaces for
> accessing features.

> * There were multiple responses around the idea of being able to

> access the intermediate index results of resampling.
>
> I would say that a majority of the responses were inline with what I
> expected...except for the few that said the documentation is great
> (I personally agree with the idea that the docs need work).
>
> The core pyresample devs met virtually a couple times to discuss
> this survey and to describe a path forward for a better Pyresample.
> We've summarized these ideas in a new "Roadmap" page on the
> documentation:
>
> https://pyresample.readthedocs.io/en/latest/roadmap.html
> <https://pyresample.readthedocs.io/en/latest/roadmap.html>
>
> I plan on working towards Pyresample 2.0 over the next 6-12 months.
> If you have any feedback on this roadmap feel free to respond to
> this mailing list post or talk to me in the "#pyresample" channel on
> the Pytroll slack (http://pytroll.github.io/#getting-in-touch

> <http://pytroll.github.io/#getting-in-touch>).

>
> Thanks to everyone who completed the survey and provided feedback.
>
> Dave
>

> --
> You received this message because you are subscribed to a topic in the
> Google Groups "pytroll" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/pytroll/7M-6mbApj8A/unsubscribe
> <https://groups.google.com/d/topic/pytroll/7M-6mbApj8A/unsubscribe>.
> To unsubscribe from this group and all its topics, send an email to
> pytroll+u...@googlegroups.com
> <mailto:pytroll+u...@googlegroups.com>.
> To view this discussion on the web, visit
> https://groups.google.com/d/msgid/pytroll/f019eb0c-27ca-4942-9d37-8af7905262ban%40googlegroups.com
> <https://groups.google.com/d/msgid/pytroll/f019eb0c-27ca-4942-9d37-8af7905262ban%40googlegroups.com?utm_medium=email&utm_source=footer>.