Task board
90 views
Skip to first unread message
Stefan van der Walt
Aug 29, 2016, 4:18:46 PM8/29/16
to scikit-image
Hi, everyone
As Juan mentioned recently, it's becoming harder to keep track of the
motion of the project as a whole. Sometimes, tickets lounge unattended
for an unacceptable long time.
I can think of two main routes for improving the situation:
1) Improve our team structure, so that individuals take responsibility
for moving certain aspects of the project forward.
2) Processes / systems to help us get a grasp on the current state of
things
I have some ideas around (1) that need fleshing out, but in the mean
time I was looking at http://zube.io to address part of (2).
Do you have any experience with this system? Zube can be compared
against HuBoard, gh-board, and ZenHub. A test scikit-image board is
here: https://zube.io/projects/4670/kanban (it's vanilla, I haven't
done anything with it yet)
Let me hear your thoughts.
Stéfan
As Juan mentioned recently, it's becoming harder to keep track of the
motion of the project as a whole. Sometimes, tickets lounge unattended
for an unacceptable long time.
I can think of two main routes for improving the situation:
1) Improve our team structure, so that individuals take responsibility
for moving certain aspects of the project forward.
2) Processes / systems to help us get a grasp on the current state of
things
I have some ideas around (1) that need fleshing out, but in the mean
time I was looking at http://zube.io to address part of (2).
Do you have any experience with this system? Zube can be compared
against HuBoard, gh-board, and ZenHub. A test scikit-image board is
here: https://zube.io/projects/4670/kanban (it's vanilla, I haven't
done anything with it yet)
Let me hear your thoughts.
Stéfan
Emmanuelle Gouillart
Aug 29, 2016, 5:38:20 PM8/29/16
to scikit...@googlegroups.com
Hi Stéfan,
thanks for continuing the discussion on this important subject. I would
reformulate your two points as 1) social aspects/community management and
2) technical tools and processes.
In my opinion, we need a little bit of both. Dashboards can be useful if
they save time and provide meaningful information that would be painful
to retrieve otherwise. One should nevertheless be wary of too many
metrics and quantitative indices (a problem that cripples the
creativity of scientific research!): it is sometimes very tempting to
merge two small PRs instead of spending time on a single difficult one.
I don't have much to say about the 2nd point since I don't know the tool.
For the 1st point, here are some ideas and suggestions from the top of my
head
- assigning / self-assigning PRs might be a good idea, as Juan suggested
- when a PR is stalling and the contributor is not responding, we should
either take over if possible, or close it after some warnings.
- keeping a good balance between the number of active contributors and
active reviewers is hard. Recently several people have contributed
several PRs, demonstrating an expertise in some domain of image
processing. Maybe we should ask them whether they would be willing to
review PRs and be more active in including these persons.
- in the core team, maybe we should say more explicitely when we are or
are not available (announce it somewhere? have a google calendar?). I
know that there are periods of time where I can't spend any time on
scikit-image, it can be weeks or even months. If the rest of the team
had such information for every core member, it would save time
because someone else could try to take over. We could also have pairs of
people with similar expertise (segmentation, computer vision,
architecture, etc.), who could work in pairs and replace the other person
when one is not available.
- we could take turns to be in charge of checking the global advance of
PRs. It's a pain of a job, but if it's one week every two months it's
not so bad.
Here are my 2 cents. I think it's a question of focussing our resources
better, and maybe empowering new people, because I'm not sure we can
spend much more time on the reviewing process.
Cheers,
Emma
thanks for continuing the discussion on this important subject. I would
reformulate your two points as 1) social aspects/community management and
2) technical tools and processes.
In my opinion, we need a little bit of both. Dashboards can be useful if
they save time and provide meaningful information that would be painful
to retrieve otherwise. One should nevertheless be wary of too many
metrics and quantitative indices (a problem that cripples the
creativity of scientific research!): it is sometimes very tempting to
merge two small PRs instead of spending time on a single difficult one.
I don't have much to say about the 2nd point since I don't know the tool.
For the 1st point, here are some ideas and suggestions from the top of my
head
- assigning / self-assigning PRs might be a good idea, as Juan suggested
- when a PR is stalling and the contributor is not responding, we should
either take over if possible, or close it after some warnings.
- keeping a good balance between the number of active contributors and
active reviewers is hard. Recently several people have contributed
several PRs, demonstrating an expertise in some domain of image
processing. Maybe we should ask them whether they would be willing to
review PRs and be more active in including these persons.
- in the core team, maybe we should say more explicitely when we are or
are not available (announce it somewhere? have a google calendar?). I
know that there are periods of time where I can't spend any time on
scikit-image, it can be weeks or even months. If the rest of the team
had such information for every core member, it would save time
because someone else could try to take over. We could also have pairs of
people with similar expertise (segmentation, computer vision,
architecture, etc.), who could work in pairs and replace the other person
when one is not available.
- we could take turns to be in charge of checking the global advance of
PRs. It's a pain of a job, but if it's one week every two months it's
not so bad.
Here are my 2 cents. I think it's a question of focussing our resources
better, and maybe empowering new people, because I'm not sure we can
spend much more time on the reviewing process.
Cheers,
Emma
Stefan van der Walt
Aug 30, 2016, 1:33:06 PM8/30/16
to scikit...@googlegroups.com
Hi Emmanuelle
On Mon, Aug 29, 2016, at 14:38, Emmanuelle Gouillart wrote:
> In my opinion, we need a little bit of both. Dashboards can be useful if
> they save time and provide meaningful information that would be painful
> to retrieve otherwise. One should nevertheless be wary of too many
> metrics and quantitative indices (a problem that cripples the
> creativity of scientific research!): it is sometimes very tempting to
> merge two small PRs instead of spending time on a single difficult one.
What I liked about Zube is that it has the potential to give you a good
"bird's eye view" of PRs. We could probably achieve the same using
(tags + some script), or (tags + gh-board). But it's good to know
what's being worked on, what's on the back-burner, etc. in one glance.
I don't care too much about metrics and burndown charts etc., which I
think are less applicable to a community driven project.
> - when a PR is stalling and the contributor is not responding, we should
> either take over if possible, or close it after some warnings.
We should feel confident to triage more aggressively. Open tickets that
hang around also sap resources, in making it harder to find the right
things to work on when you only have a little bit of time.
> - in the core team, maybe we should say more explicitely when we are or
> are not available (announce it somewhere? have a google calendar?).
I think this ties in with your next point. It's hard to track
calendars; perhaps more practical is to track commitment? There is
already a (dangerous) notion of "first person to comment essentially
owns the review for the PR", but we should make it more explicit: both
so that folks don't feel afraid to review PRs ("oh no, I'm becoming
responsible for this thing!") and so that there is someone actively
watching over its progress. That way, we can also easily find PRs that
have no "owners".
> - we could take turns to be in charge of checking the global advance of
> PRs. It's a pain of a job, but if it's one week every two months it's
> not so bad.
I like this idea; I've been wondering how we could move "ownership" of
the project around to create better focus and more autonomy, and this
may be a good, gentle start to that.
I'd be curious how others felt about this idea?
> Here are my 2 cents. I think it's a question of focussing our resources
> better, and maybe empowering new people, because I'm not sure we can
> spend much more time on the reviewing process.
Growing the core team is crucial. Our user base is growing nicely, but
that is not helpful unless we convert a certain percentage of those
users to contributors. And it should be clear to contributors that
reviewing is a priority and a good way to become involved (everyone
wants to code, though :).
We have a document (http://scikit-image.org/docs/stable/contribute.html)
that invites users to contribute, but we can simplify those instructions
for newcomers, and also add a "Please help us review some code" button
on the front page?
Stéfan
On Mon, Aug 29, 2016, at 14:38, Emmanuelle Gouillart wrote:
> In my opinion, we need a little bit of both. Dashboards can be useful if
> they save time and provide meaningful information that would be painful
> to retrieve otherwise. One should nevertheless be wary of too many
> metrics and quantitative indices (a problem that cripples the
> creativity of scientific research!): it is sometimes very tempting to
> merge two small PRs instead of spending time on a single difficult one.
"bird's eye view" of PRs. We could probably achieve the same using
(tags + some script), or (tags + gh-board). But it's good to know
what's being worked on, what's on the back-burner, etc. in one glance.
I don't care too much about metrics and burndown charts etc., which I
think are less applicable to a community driven project.
> - when a PR is stalling and the contributor is not responding, we should
> either take over if possible, or close it after some warnings.
hang around also sap resources, in making it harder to find the right
things to work on when you only have a little bit of time.
> - in the core team, maybe we should say more explicitely when we are or
> are not available (announce it somewhere? have a google calendar?).
calendars; perhaps more practical is to track commitment? There is
already a (dangerous) notion of "first person to comment essentially
owns the review for the PR", but we should make it more explicit: both
so that folks don't feel afraid to review PRs ("oh no, I'm becoming
responsible for this thing!") and so that there is someone actively
watching over its progress. That way, we can also easily find PRs that
have no "owners".
> - we could take turns to be in charge of checking the global advance of
> PRs. It's a pain of a job, but if it's one week every two months it's
> not so bad.
the project around to create better focus and more autonomy, and this
may be a good, gentle start to that.
I'd be curious how others felt about this idea?
> Here are my 2 cents. I think it's a question of focussing our resources
> better, and maybe empowering new people, because I'm not sure we can
> spend much more time on the reviewing process.
that is not helpful unless we convert a certain percentage of those
users to contributors. And it should be clear to contributors that
reviewing is a priority and a good way to become involved (everyone
wants to code, though :).
We have a document (http://scikit-image.org/docs/stable/contribute.html)
that invites users to contribute, but we can simplify those instructions
for newcomers, and also add a "Please help us review some code" button
on the front page?
Stéfan
Johannes Schönberger
Aug 30, 2016, 3:58:47 PM8/30/16
to scikit...@googlegroups.com
Hi,
Please, find my comments inline.
>> In my opinion, we need a little bit of both. Dashboards can be useful if
>> they save time and provide meaningful information that would be painful
>> to retrieve otherwise. One should nevertheless be wary of too many
>> metrics and quantitative indices (a problem that cripples the
>> creativity of scientific research!): it is sometimes very tempting to
>> merge two small PRs instead of spending time on a single difficult one.
>
> What I liked about Zube is that it has the potential to give you a good
> "bird's eye view" of PRs. We could probably achieve the same using
> (tags + some script), or (tags + gh-board). But it's good to know
> what's being worked on, what's on the back-burner, etc. in one glance.
> I don't care too much about metrics and burndown charts etc., which I
> think are less applicable to a community driven project.
I agree that Github does not provide a good overview of active/passive issues. I am overwhelmed by the amount of issues and, usually, after a week of scikit-image "abstinence" :-), I have to start from scratch to find the relevant issues. Tags don't really help me in this regard either...
>> - when a PR is stalling and the contributor is not responding, we should
>> either take over if possible, or close it after some warnings.
>
> We should feel confident to triage more aggressively. Open tickets that
> hang around also sap resources, in making it harder to find the right
> things to work on when you only have a little bit of time.
+1, there should be a cleanup of old issues that are stale for months if not years.
+1, In my opinion, reviewing is almost more important than writing the code. With the current system, Github only "rewards" people who contribute code and, I think, this is part of the problem. With every commit, people get credit in the form of # commits or # lines of code and Github shows the stats nicely in the profile or the project graphs. Maybe there is a way to create an equivalent for the reviewing process, e.g., # comments and # merged PRs.
What do people think about this idea?
Best,
Johannes
Please, find my comments inline.
>> In my opinion, we need a little bit of both. Dashboards can be useful if
>> they save time and provide meaningful information that would be painful
>> to retrieve otherwise. One should nevertheless be wary of too many
>> metrics and quantitative indices (a problem that cripples the
>> creativity of scientific research!): it is sometimes very tempting to
>> merge two small PRs instead of spending time on a single difficult one.
>
> What I liked about Zube is that it has the potential to give you a good
> "bird's eye view" of PRs. We could probably achieve the same using
> (tags + some script), or (tags + gh-board). But it's good to know
> what's being worked on, what's on the back-burner, etc. in one glance.
> I don't care too much about metrics and burndown charts etc., which I
> think are less applicable to a community driven project.
>> - when a PR is stalling and the contributor is not responding, we should
>> either take over if possible, or close it after some warnings.
>
> We should feel confident to triage more aggressively. Open tickets that
> hang around also sap resources, in making it harder to find the right
> things to work on when you only have a little bit of time.
What do people think about this idea?
Best,
Johannes
Stefan van der Walt
Aug 30, 2016, 7:54:11 PM8/30/16
to scikit...@googlegroups.com
Hi Johannes
On Tue, Aug 30, 2016, at 12:58, Johannes Schönberger wrote:
> I agree that Github does not provide a good overview of active/passive
> issues. I am overwhelmed by the amount of issues and, usually, after a
> week of scikit-image "abstinence" :-), I have to start from scratch to
> find the relevant issues. Tags don't really help me in this regard
> either...
Did you look at the proposed Zube (or the alternatives)? I'd be curious
to hear what you think.
> > We have a document (http://scikit-image.org/docs/stable/contribute.html)
> > that invites users to contribute, but we can simplify those instructions
> > for newcomers, and also add a "Please help us review some code" button
> > on the front page?
>
> +1, In my opinion, reviewing is almost more important than writing the
> code. With the current system, Github only "rewards" people who
> contribute code and, I think, this is part of the problem. With every
> commit, people get credit in the form of # commits or # lines of code and
> Github shows the stats nicely in the profile or the project graphs. Maybe
> there is a way to create an equivalent for the reviewing process, e.g., #
> comments and # merged PRs.
>
> What do people think about this idea?
We can extend our release script to pull that information from GitHub.
Would it be helpful to credit with new releases, or should this come as
more continuous feedback, e.g. a weekly mail to the list with
statistics?
Stéfan
On Tue, Aug 30, 2016, at 12:58, Johannes Schönberger wrote:
> I agree that Github does not provide a good overview of active/passive
> issues. I am overwhelmed by the amount of issues and, usually, after a
> week of scikit-image "abstinence" :-), I have to start from scratch to
> find the relevant issues. Tags don't really help me in this regard
> either...
to hear what you think.
> > We have a document (http://scikit-image.org/docs/stable/contribute.html)
> > that invites users to contribute, but we can simplify those instructions
> > for newcomers, and also add a "Please help us review some code" button
> > on the front page?
>
> +1, In my opinion, reviewing is almost more important than writing the
> code. With the current system, Github only "rewards" people who
> contribute code and, I think, this is part of the problem. With every
> commit, people get credit in the form of # commits or # lines of code and
> Github shows the stats nicely in the profile or the project graphs. Maybe
> there is a way to create an equivalent for the reviewing process, e.g., #
> comments and # merged PRs.
>
> What do people think about this idea?
Would it be helpful to credit with new releases, or should this come as
more continuous feedback, e.g. a weekly mail to the list with
statistics?
Stéfan
Egor Panfilov
Sep 2, 2016, 4:55:02 AM9/2/16
to scikit...@googlegroups.com
Hello everyone!
Here are some of mine reflections on the topic:
Regarding the points raised by Stefan, I think that the main issue we have is a lack of resources and reviews (just a casual bump of https://github.com/scikit-image/scikit-image/pull/2040 here ;) ). I understand that everyone in the core team has a lot of matters to deal with outside the project, but the activity from our (maintainers) side is far less compared to the activity of the contributors. In my opinion, people tend to quit contributing and even stop using OSS if they don't have enough treatment. Of course, many of the PRs we currently have are quite poor in terms of quiality and/or abandoned (we have to be more aggressive managing them), but those which are pretty good, maybe not perfect, totally deserve to get much more attention (e.g. https://github.com/scikit-image/scikit-image/pull/1957, https://github.com/scikit-image/scikit-image/pull/1570, etc). I'd prefer to have slow (pure Python) implementations of established image processing algorithms and make more people involved in the project rather then asking them to write an extremely fast and ideal code and losing them in process. So, pt.1 - "we should do more and grow the active community".
As for issues/PRs management system, I think that GitHub tags pretty much do the deal. We've recently added "status: xxx" tags to facilitate tracking of active/abandoned/WIP/review+1 issues/PRs. The only _extra_ feature I'd personally prefer to have is issues/PRs aging, so to easily observe which of them haven't gotten any activity recently. If the management system offers this kind of functionality, I'd be glad to give a shot testing it, otherwise I'm neutral about this.
From the pricing perspective, Zube is not the best choice as it offers Free plans for teams up to 4 people. I think that https://waffle.io/ could be a better alternative. Here you can see a preview https://waffle.io/scikit-image/scikit-image . From my knowledge, Waffle is one of the first services of this kind, so should be well-designed from the UI perspective.
If we'd like not just to manage open GitHub issues/PRs, but have a more structured backlog for our discussions and RFCs (I'd __love__ to have RFCs, not just random discussions in issues/mailing list), Trello (trello.com) could be also a nice choice. It has awesome GitHub integration and many more features. Although, I'm not sure that devs would like to get acquainted with this heavy system just for a single project management.
Pt.2 - "issues/PRs management tool for stronger management, not just for its own sake".
P.S. I like the practice of `scikit-learn` guys to rename PRs as "[MRG] some awesome contribution" -> [MRG + 1] some awesome contribution". Pretty simple and cheap solution to track LGTM :+1: entities.
One more point (yet it is better to start a discussion in a RFC :) ) I'd like to bring to the discussion. I remember a short discussion with Stefan, that there was a decision upon that `skimage` is a community driven-project. I understand, that it is probably mostly used for educational purposes, but we shouldn't as a project fall behind the progress in Image Processing and Computer Vision. I see that many modern Deep Learning frameworks start to implement their own image processing routines just to have a common ground for their functionality. I believe we could still catch the train. So, I propose to consider moving `skimage` infrastructure onto `theano`. That is a hell lot of work, of course, but I'm asking just for a discussion yet.
From this section, actually, one more pragmatic topic arises. Even if we keep image processing part up to the community, I think that we have to put more our (core team) efforts on the backend part of the project (not just testing and documentation part, but also e.g. make easier function chaining - https://github.com/scikit-image/scikit-image/issues/1529). I'd not expect from anyone outside the core team to have enough knowledge, experience and confidence to implement this and some other features. I.e. I believe that we have to make some necessary contributions to stay competitive.
+1, In my opinion, reviewing is almost more important than writing the code. With the current system, Github only "rewards" people who contribute code and, I think, this is part of the problem. With every commit, people get credit in the form of # commits or # lines of code and Github shows the stats nicely in the profile or the project graphs. Maybe there is a way to create an equivalent for the reviewing process, e.g., # comments and # merged PRs.
I fully agree with you, Johannes, but have not a single idea on how to implement this. Well, we still can set up a GitHub bot (sorry, I'm being unreasonably hyped by this tech these days) and track who participated in the review process for each PR, but I'm definitely not sure how fair it could be (what if I just comment something like "LOL" in 1e6 lines of code PR and got credited for that) and what kind of appreciation one might expect (like, "this person has reviewed a lot"). Ofc, we can track who put "LGTM" or "+1" on the PR page, but this isn't a fair solution either. We are already publishing "list of fame" with every new release, and an organization badge in GitHub account is self-speaking. I guess that's the most safe option we could have at the moment.
Sorry for the long read. :)
Have a nice we.
Regards,
Egor Panfilov
--
You received this message because you are subscribed to the Google Groups "scikit-image" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image+unsubscribe@googlegroups.com.
To post to this group, send an email to scikit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/1472601249.1617176.710940585.4C1ED035%40webmail.messagingengine.com.
For more options, visit https://groups.google.com/d/optout.
Emmanuelle Gouillart
Sep 2, 2016, 8:48:14 AM9/2/16
to scikit...@googlegroups.com
[changing the title of the thread since one of your comments deviated
quite a lot from the topic of "task board"]
> One more point (yet it is better to start a discussion in a RFC :) ) I'd like
> to bring to the discussion. I remember a short discussion with Stefan, that
> there was a decision upon that `skimage` is a community driven-project. I
> understand, that it is probably mostly used for educational purposes, but we
> shouldn't as a project fall behind the progress in Image Processing and
> Computer Vision. I see that many modern Deep Learning frameworks start to
> implement their own image processing routines just to have a common ground for
> their functionality. I believe we could still catch the train. So, I propose to
> consider moving `skimage` infrastructure onto `theano`. That is a hell lot of
> work, of course, but I'm asking just for a discussion yet.
> From this section, actually, one more pragmatic topic arises. Even if we keep
> image processing part up to the community, I think that we have to put more our
> (core team) efforts on the backend part of the project (not just testing and
> documentation part, but also e.g. make easier function chaining - https://
> github.com/scikit-image/scikit-image/issues/1529). I'd not expect from anyone
> outside the core team to have enough knowledge, experience and confidence to
> implement this and some other features. I.e. I believe that we have to make
> some necessary contributions to stay competitive.
I have to say that I disagree a lot with this analysis.
1. First of all, I don't think that scikit-image is used primarily for
educational purposes. If you browse through the research articles that
cite the scikit-image paper, you'll find that people have used
scikit-image as a building block for scientific software and applications
or for research in various fields such as medical science, materials
science, plant research, etc. I know several industrial groups which are
using scikit-image for their daily R&D workflow. Tens of thousands of
people are browsing every month through the examples of the gallery to
discover how to perform specific image processing operations.
That said, you're totally right to raise the question of "staying
competitive". But it's a tricky issue, since in my opinion, being
competitive is a subtle mixture of usability, features, performance
(probably in this order, but it's a matter of debate).
2. About Deep Learning: deep learning is very cool, trendy and powerful,
it can blow your mind for some operations, yet it cannot do everything in
image processing. Labeling images with kittens is not the sole purpose
that our users have :-). Furthermore, deep learning require intensive
computations, that are best done on the GPU. Some packages like keras
provide elegant solutions for deep learning and are compatible with
scikit-image, thanks to the common use of numpy arrays.
Can some scikit-image contributors work on a deep learning package for
image processing using theano, tensorflow, keras and some of the
knowledge/features of scikit-image? Probably. Does it have to be in
scikit-image? I don't think so (although it could be a project of the
scikit-image organization). Let's try not to build a monster package with
everything and the kitchen sink, that quickly becomes uninstallable and
unusable.
Using theano instead of NumPy arrays would be to favor one image
processing workflow at the expense of many others, so I don't think it's
wise. To quote David Waley-Farley: "Yep. Theano, Numba, numexpr, Cython
and PyPy shall one day all merge to form Numtron, defender of sanity,
runner of fast numerics.". But before we're there, NumPy is the common
denominator for the scientific packages.
It would be interesting to know which are the deep learning frameworks
that you're mentioning, to see how they tackle image processing. I'd be
interested in knowing more about this.
3. For function chaining, it might be interesting to take a look at
scikit-learn's pipeline
http://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html
My 2 cents,
Emma
quite a lot from the topic of "task board"]
> One more point (yet it is better to start a discussion in a RFC :) ) I'd like
> to bring to the discussion. I remember a short discussion with Stefan, that
> there was a decision upon that `skimage` is a community driven-project. I
> understand, that it is probably mostly used for educational purposes, but we
> shouldn't as a project fall behind the progress in Image Processing and
> Computer Vision. I see that many modern Deep Learning frameworks start to
> implement their own image processing routines just to have a common ground for
> their functionality. I believe we could still catch the train. So, I propose to
> consider moving `skimage` infrastructure onto `theano`. That is a hell lot of
> work, of course, but I'm asking just for a discussion yet.
> From this section, actually, one more pragmatic topic arises. Even if we keep
> image processing part up to the community, I think that we have to put more our
> (core team) efforts on the backend part of the project (not just testing and
> documentation part, but also e.g. make easier function chaining - https://
> github.com/scikit-image/scikit-image/issues/1529). I'd not expect from anyone
> outside the core team to have enough knowledge, experience and confidence to
> implement this and some other features. I.e. I believe that we have to make
> some necessary contributions to stay competitive.
1. First of all, I don't think that scikit-image is used primarily for
educational purposes. If you browse through the research articles that
cite the scikit-image paper, you'll find that people have used
scikit-image as a building block for scientific software and applications
or for research in various fields such as medical science, materials
science, plant research, etc. I know several industrial groups which are
using scikit-image for their daily R&D workflow. Tens of thousands of
people are browsing every month through the examples of the gallery to
discover how to perform specific image processing operations.
That said, you're totally right to raise the question of "staying
competitive". But it's a tricky issue, since in my opinion, being
competitive is a subtle mixture of usability, features, performance
(probably in this order, but it's a matter of debate).
2. About Deep Learning: deep learning is very cool, trendy and powerful,
it can blow your mind for some operations, yet it cannot do everything in
image processing. Labeling images with kittens is not the sole purpose
that our users have :-). Furthermore, deep learning require intensive
computations, that are best done on the GPU. Some packages like keras
provide elegant solutions for deep learning and are compatible with
scikit-image, thanks to the common use of numpy arrays.
Can some scikit-image contributors work on a deep learning package for
image processing using theano, tensorflow, keras and some of the
knowledge/features of scikit-image? Probably. Does it have to be in
scikit-image? I don't think so (although it could be a project of the
scikit-image organization). Let's try not to build a monster package with
everything and the kitchen sink, that quickly becomes uninstallable and
unusable.
Using theano instead of NumPy arrays would be to favor one image
processing workflow at the expense of many others, so I don't think it's
wise. To quote David Waley-Farley: "Yep. Theano, Numba, numexpr, Cython
and PyPy shall one day all merge to form Numtron, defender of sanity,
runner of fast numerics.". But before we're there, NumPy is the common
denominator for the scientific packages.
It would be interesting to know which are the deep learning frameworks
that you're mentioning, to see how they tackle image processing. I'd be
interested in knowing more about this.
3. For function chaining, it might be interesting to take a look at
scikit-learn's pipeline
http://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.html
My 2 cents,
Emma
Stefan van der Walt
Sep 2, 2016, 7:59:06 PM9/2/16
to scikit...@googlegroups.com
Hi Egor
On Fri, Sep 2, 2016, at 01:54, Egor Panfilov wrote:
Regarding the points raised by Stefan, I think that the main issue we have is a lack of resources and reviews (just a casual bump of https://github.com/scikit-image/scikit-image/pull/2040 here ;) ).
I still don't see any explanation of what the term 'l2hys' means!
I understand that everyone in the core team has a lot of matters to deal with outside the project, but the activity from our (maintainers) side is far less compared to the activity of the contributors. In my opinion, people tend to quit contributing
This is true, unfortunately. How do we increase the number of hours available? I can see two ways: grow the contributor community (in such a way that contributors know to review as well) or hire someone to work on the project full time. I'd be interested in exploring the latter avenue. We can also reach out to employers of existing contributors and try and negotiated dedicated working time on the project.
As for issues/PRs management system, I think that GitHub tags pretty much do the deal. We've recently added "status: xxx" tags to facilitate tracking of active/abandoned/WIP/review+1 issues/PRs.
The issue I wanted to address was getting a good "bird's eye view" on issues. I don't think tags are that useful in that regard.
From the pricing perspective, Zube is not the best choice as it offers Free plans for teams up to 4 people. I think
No, Zube is free for open source, for unlimited users.
(I'd __love__ to have RFCs, not just random discussions in issues/mailing list), Trello (trello.com) could be also a nice choice.
You mean RFCs like PEPs?
Pt.2 - "issues/PRs management tool for stronger management, not just for its own sake".P.S. I like the practice of `scikit-learn` guys to rename PRs as "[MRG] some awesome contribution" -> [MRG + 1] some awesome contribution". Pretty simple and cheap solution to track LGTM :+1: entities.
I'd be fine with that.
outines just to have a common ground for their functionality. I believe we could still catch the train. So, I propose to consider moving `skimage` infrastructure onto `theano`. That is a hell lot of work, of course, but I'm asking just for a discussion yet.
We do evaluate these things from time to time. When GPUs came into fashion, we looked at those, at a recent conference I spoke to folks about using DSLs like Halide for speed optimization, and with our GSoC student Daniil we've spoken a lot about integrating neural nets. Adopting theano may be quite tricky, so perhaps a proof of concept (or an RFC ;) would be the easiest way to kick off that conversation.
From this section, actually, one more pragmatic topic arises. Even if we keep image processing part up to the community, I think that we have to put more our (core team) efforts on the backend part of the project (not just testing and documentation part, but also e.g. make easier function chaining - https://github.com/scikit-image/scikit-image/issues/1529). I'd not expect from anyone outside the core team to have enough knowledge, experience and confidence to implement this and some other features. I.e. I believe that we have to make some necessary contributions to stay competitive.
Can you explain what you mean?
Thanks for all your comments,
Stéfan
Johannes Schönberger
Sep 3, 2016, 5:32:11 AM9/3/16
to scikit...@googlegroups.com
Hi,
>> I understand that everyone in the core team has a lot of matters to deal with outside the project, but the activity from our (maintainers) side is far less compared to the activity of the contributors. In my opinion, people tend to quit contributing
>
> This is true, unfortunately. How do we increase the number of hours available? I can see two ways: grow the contributor community (in such a way that contributors know to review as well) or hire someone to work on the project full time. I'd be interested in exploring the latter avenue. We can also reach out to employers of existing contributors and try and negotiated dedicated working time on the project.
The latter would be a great step forward!
>> Pt.2 - "issues/PRs management tool for stronger management, not just for its own sake".
>> P.S. I like the practice of `scikit-learn` guys to rename PRs as "[MRG] some awesome contribution" -> [MRG + 1] some awesome contribution". Pretty simple and cheap solution to track LGTM :+1: entities.
>
> I'd be fine with that.
+1
>> outines just to have a common ground for their functionality. I believe we could still catch the train. So, I propose to consider moving `skimage` infrastructure onto `theano`. That is a hell lot of work, of course, but I'm asking just for a discussion yet.
>
> We do evaluate these things from time to time. When GPUs came into fashion, we looked at those, at a recent conference I spoke to folks about using DSLs like Halide for speed optimization, and with our GSoC student Daniil we've spoken a lot about integrating neural nets. Adopting theano may be quite tricky, so perhaps a proof of concept (or an RFC ;) would be the easiest way to kick off that conversation.
I'd argue that scikit-image wouldn't gain any more traction by transitioning to Theano. The big advantage of theano in the deep learning setting is its automatic differentiation functionality. This will only be useful for a very small subset of functions in scikit-image. The input/output variables to any Python-based deep learning framework are still numpy arrays.
>> From this section, actually, one more pragmatic topic arises. Even if we keep image processing part up to the community, I think that we have to put more our (core team) efforts on the backend part of the project (not just testing and documentation part, but also e.g. make easier function chaining - https://github.com/scikit-image/scikit-image/issues/1529). I'd not expect from anyone outside the core team to have enough knowledge, experience and confidence to implement this and some other features. I.e. I believe that we have to make some necessary contributions to stay competitive.
It would be good, if you could elaborate a little more on this point. Most of skimage's API was designed with exactly that in mind.
Best,
Johannes
>> I understand that everyone in the core team has a lot of matters to deal with outside the project, but the activity from our (maintainers) side is far less compared to the activity of the contributors. In my opinion, people tend to quit contributing
>
> This is true, unfortunately. How do we increase the number of hours available? I can see two ways: grow the contributor community (in such a way that contributors know to review as well) or hire someone to work on the project full time. I'd be interested in exploring the latter avenue. We can also reach out to employers of existing contributors and try and negotiated dedicated working time on the project.
>> Pt.2 - "issues/PRs management tool for stronger management, not just for its own sake".
>> P.S. I like the practice of `scikit-learn` guys to rename PRs as "[MRG] some awesome contribution" -> [MRG + 1] some awesome contribution". Pretty simple and cheap solution to track LGTM :+1: entities.
>
> I'd be fine with that.
>> outines just to have a common ground for their functionality. I believe we could still catch the train. So, I propose to consider moving `skimage` infrastructure onto `theano`. That is a hell lot of work, of course, but I'm asking just for a discussion yet.
>
> We do evaluate these things from time to time. When GPUs came into fashion, we looked at those, at a recent conference I spoke to folks about using DSLs like Halide for speed optimization, and with our GSoC student Daniil we've spoken a lot about integrating neural nets. Adopting theano may be quite tricky, so perhaps a proof of concept (or an RFC ;) would be the easiest way to kick off that conversation.
>> From this section, actually, one more pragmatic topic arises. Even if we keep image processing part up to the community, I think that we have to put more our (core team) efforts on the backend part of the project (not just testing and documentation part, but also e.g. make easier function chaining - https://github.com/scikit-image/scikit-image/issues/1529). I'd not expect from anyone outside the core team to have enough knowledge, experience and confidence to implement this and some other features. I.e. I believe that we have to make some necessary contributions to stay competitive.
Best,
Johannes
Egor Panfilov
Sep 4, 2016, 4:32:23 AM9/4/16
to scikit...@googlegroups.com
Hi Stefan!
As for issues/PRs management system, I think that GitHub tags pretty much do the deal. We've recently added "status: xxx" tags to facilitate tracking of active/abandoned/WIP/review+1 issues/PRs.The issue I wanted to address was getting a good "bird's eye view" on issues. I don't think tags are that useful in that regard.
Yes, I see. I think, this could be helpful for project owners to make strategic decisions. On the other hand, I personally don't find a great benefit of using these tools for triaging, cross-linking issues/PRs and other cases I care about. Maybe we should just give it a try.
From the pricing perspective, Zube is not the best choice as it offers Free plans for teams up to 4 people. I thinkNo, Zube is free for open source, for unlimited users.
Indeed, sorry, I see this now.
(I'd __love__ to have RFCs, not just random discussions in issues/mailing list), Trello (trello.com) could be also a nice choice.You mean RFCs like PEPs?
Exactly. I think, this format would help to accumulate our opinions on different matters (e.g. how to we treat and depend on `matplotlib`), for which website (deeply buried pages of it) is not the best place. At the moment, discussions on many sensitive topics appear again and again in the different PRs, making them hard to align and to be referenced.
As for technical side of this, we probably could reuse GitHub issues (by creating exclusive tag "SkimageEP" or something similar). `matplotlib` people, as far as I know, use GitHub Wiki to post "xEP"s, but I'm not sure where do they held the discusssions. GitHub does also support protecting different pages from modifcation, that could also be useful.
From this section, actually, one more pragmatic topic arises. Even if we keep image processing part up to the community, I think that we have to put more our (core team) efforts on the backend part of the project (not just testing and documentation part, but also e.g. make easier function chaining - https://github.com/scikit-image/scikit-image/issues/1529). I'd not expect from anyone outside the core team to have enough knowledge, experience and confidence to implement this and some other features. I.e. I believe that we have to make some necessary contributions to stay competitive.Can you explain what you mean?
Well, I'm just saying that we as a core team should drive the evolution of backend and make life easier for contributors and users. Like, try to imagine some new contributor making a PR introducing Sphinx Gallery. Can't see this happenning. Of course, all of us are working on this kind of features already, I just want to highlight this (maybe trivial) point.
As a "practical guide", perhaps we should dedicate more time to the _complex_ issues/requests/PRs we have.
Thanks for all your comments on Theano/GPU/whatever part (/cc Emmanuelle, Johannes), they're very informative and insightful. I'd prefer to raise this discussion again (as I have more food to bring to the table) or maybe just collect your opinions in one place when (and if) we decipe on SkimageEPs.
Regards,
Egor
Johannes Schönberger
Sep 4, 2016, 4:46:32 AM9/4/16
to scikit...@googlegroups.com
To better keep track of PR/issues, I suggest core developers assign themselves to some of the issues in their field of expertise. There are so many PRs that are >90% finished but it seems we forgot to continue reviewing them.
> To post to this group, send email to scikit...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/CAP0v5tm5C4%3Df9c9u1juf37yYpXipkZU%3Ds1zwVO5GMikqnB2KVQ%40mail.gmail.com.
> --
> You received this message because you are subscribed to the Google Groups "scikit-image" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image...@googlegroups.com.
> You received this message because you are subscribed to the Google Groups "scikit-image" group.
> To post to this group, send email to scikit...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/CAP0v5tm5C4%3Df9c9u1juf37yYpXipkZU%3Ds1zwVO5GMikqnB2KVQ%40mail.gmail.com.
Emmanuelle Gouillart
Sep 4, 2016, 4:51:23 AM9/4/16
to scikit...@googlegroups.com
On Sun, Sep 04, 2016 at 10:46:28AM +0200, Johannes Schönberger wrote:
> To better keep track of PR/issues, I suggest core developers assign themselves to some of the issues in their field of expertise. There are so many PRs that are >90% finished but it seems we forgot to continue reviewing them.
Yes, and we can also tentatively assign someone else if we feel that a
> To better keep track of PR/issues, I suggest core developers assign themselves to some of the issues in their field of expertise. There are so many PRs that are >90% finished but it seems we forgot to continue reviewing them.
given PR is in his/her realm of expertise. If the other person disagrees,
it's always possible to un-assign oneself :-).
Johannes Schönberger
Sep 4, 2016, 4:56:01 AM9/4/16
to scikit...@googlegroups.com
>> To better keep track of PR/issues, I suggest core developers assign themselves to some of the issues in their field of expertise. There are so many PRs that are >90% finished but it seems we forgot to continue reviewing them.
>
> Yes, and we can also tentatively assign someone else if we feel that a
> given PR is in his/her realm of expertise. If the other person disagrees,
> it's always possible to un-assign oneself :-).
>
> --
> You received this message because you are subscribed to the Google Groups "scikit-image" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/20160904085122.GA309739%40phare.normalesup.org.
Stefan van der Walt
Sep 5, 2016, 6:42:50 AM9/5/16
to scikit...@googlegroups.com, Ralf Gommers
On Sun, Sep 4, 2016, at 01:32, Egor Panfilov wrote:
You mean RFCs like PEPs?Exactly. I think, this format would help to accumulate our opinions on different matters (e.g. how to we treat and depend on `matplotlib`)
We should chat to Ralf Gommers to find out how successful the Enhancement Proposal approach has been for SciPy, which is similar to this project in many ways.
Well, I'm just saying that we as a core team should drive the evolution of backend and make life easier for contributors and users. Like, try to imagine some new contributor making a PR introducing Sphinx Gallery. Can't see this happenning. Of course, all of us are working on this kind of features already, I just want to highlight this (maybe trivial) point.As a "practical guide", perhaps we should dedicate more time to the _complex_ issues/requests/PRs we have.
Still, I'm unclear how we can make it easier for newcomers to contribute to the backend. Do you have any practical suggestions?
Best regards
Stefan
Ralf Gommers
Sep 5, 2016, 4:05:42 PM9/5/16
to scikit...@googlegroups.com
On Mon, Sep 5, 2016 at 10:42 PM, Stefan van der Walt <ste...@berkeley.edu> wrote:
On Sun, Sep 4, 2016, at 01:32, Egor Panfilov wrote:You mean RFCs like PEPs?Exactly. I think, this format would help to accumulate our opinions on different matters (e.g. how to we treat and depend on `matplotlib`)We should chat to Ralf Gommers to find out how successful the Enhancement Proposal approach has been for SciPy, which is similar to this project in many ways.
Since we don't have those for SciPy, I'd say not very:) You're thinking about NumPy probably:
http://docs.scipy.org/doc/numpy/neps/index.html
https://github.com/numpy/numpy/tree/master/doc/neps
http://docs.scipy.org/doc/numpy/neps/index.html
https://github.com/numpy/numpy/tree/master/doc/neps
We don't use them often, but when we do they've been valuable. For example the .npy version format is a good reference with rationale, and for complex discussions like datetime and NA it helps to summarize the issues and outlike solution directions (emails really don't work well for that).
Cheers,
Ralf
Stefan van der Walt
Sep 14, 2016, 7:08:58 PM9/14/16
to scikit-image
On Mon, Aug 29, 2016, at 13:17, Stefan van der Walt wrote:
> I have some ideas around (1) that need fleshing out, but in the mean
> time I was looking at http://zube.io to address part of (2).
Take a look at this!
> I have some ideas around (1) that need fleshing out, but in the mean
> time I was looking at http://zube.io to address part of (2).
https://github.com/blog/2256-a-whole-new-github-universe-announcing-new-tools-forums-and-features
E.g., here's an empty project to play around with:
https://github.com/scikit-image/scikit-image/projects/1
Stéfan
Juan Nunez-Iglesias
Sep 15, 2016, 9:29:26 AM9/15/16
to scikit...@googlegroups.com
Yes, crazy timing! =D
--
You received this message because you are subscribed to the Google Groups "scikit-image" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image...@googlegroups.com.
To post to this group, send an email to scikit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/1473894533.1455710.726083169.1D4C654B%40webmail.messagingengine.com.
François Boulogne
Sep 16, 2016, 4:36:47 AM9/16/16
to scikit...@googlegroups.com
I'm just discovering this new feature.
It looks like we need to organize ourself because sorting issues in the
column of a project is like duplicating the labels "status" we have on
issues.
I didn't find a way to automatically import issues, but we can search a
label and drag the "cards".
I think we need to determine how many boards we want (what github call
project, I don't like this word... anyway).
A big one with the progress or small ones, one per submodule? or one per
core dev?
Any thoughts?
It looks like we need to organize ourself because sorting issues in the
column of a project is like duplicating the labels "status" we have on
issues.
I didn't find a way to automatically import issues, but we can search a
label and drag the "cards".
I think we need to determine how many boards we want (what github call
project, I don't like this word... anyway).
A big one with the progress or small ones, one per submodule? or one per
core dev?
Any thoughts?
Stefan van der Walt
Sep 20, 2016, 1:34:55 PM9/20/16
to François Boulogne, scikit...@googlegroups.com
On Fri, Sep 16, 2016, at 01:35, François Boulogne wrote:
> I think we need to determine how many boards we want (what github call
> project, I don't like this word... anyway).
> A big one with the progress or small ones, one per submodule? or one per
> core dev?
I think each board is associated with an outcome. So, e.g., we will
> I think we need to determine how many boards we want (what github call
> project, I don't like this word... anyway).
> A big one with the progress or small ones, one per submodule? or one per
> core dev?
decide on some projects such as "Improve Documentation", or "Add 3D
capability", or "Catch up on review backlog". Each of those will then
have a few standard columns that we have to decide on.
An initial suggestion would be:
Inbox (list all relevant PRs)
In Progress (someone is working on it)
Ready for Review (please review this work)
Done
Would that cover 80% of use cases, or do we need more columns?
Stéfan
François Boulogne
Sep 20, 2016, 2:17:08 PM9/20/16
to scikit...@googlegroups.com
> I think each board is associated with an outcome. So, e.g., we will
> decide on some projects such as "Improve Documentation", or "Add 3D
> capability", or "Catch up on review backlog". Each of those will then
> have a few standard columns that we have to decide on.
> An initial suggestion would be:
>
> Inbox (list all relevant PRs)
> In Progress (someone is working on it)
> Ready for Review (please review this work)
> Done
>
> Would that cover 80% of use cases, or do we need more columns?
Maybe we should had Needs Decision. Often, we have tickets stuck on a
>
> Inbox (list all relevant PRs)
> In Progress (someone is working on it)
> Ready for Review (please review this work)
> Done
>
> Would that cover 80% of use cases, or do we need more columns?
decision to take, either on new features (names, API...) or on
refactoring (change API, dependencies, add new functions, reorganize a
code...). These choices are often not straightforward and we need
different opinions.
--
François Boulogne.
http://www.sciunto.org
GPG: 32D5F22F
Johannes Schönberger
Sep 20, 2016, 3:56:43 PM9/20/16
to scikit...@googlegroups.com
This sounds good to me and we can add more columns as needed and as we gain some experience with the workflow. Grouping comments into reviews and then having a separate approval status is already a big step forward in my view.
Stéfan
--
You received this message because you are subscribed to the Google Groups "scikit-image" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scikit-image...@googlegroups.com.
To post to this group, send an email to scikit...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/scikit-image/1474392890.2276972.731679577.39AE1588%40webmail.messagingengine.com.
Reply all
Reply to author
Forward
0 new messages