Project management and Epics and Dependencies

[Thad Guidry]

Hi Dev Team!

And I'm still disgusted that GitHub does not have good project management features like Jira or Phabricator to see and plan a roadmap with dependencies.  But the good news is that GitHub support recommends ZenHub that does just that and more and supports dependencies and it's all right within GitHub's UI.  Azure Boards doesn't integrate or sync with GitHub issues, I tried it and also saw their roadmap.

I would like us to enable ZenHub (free for us since we are open source) on the OpenRefine org in GitHub and then we could make those dependencies and have real EPICs (instead of the fake ones I've been creating), and then everyone can participate with their same GitHub login.

This benefits us in lots of ways.  Unlike Phabricator or Jira, no hosting needing, 100% free for us, and no new application to learn, it's native inside of GitHub and simply enables extra features.

Take a look: https://help.zenhub.com/support/solutions/articles/43000010349-create-github-issue-dependencies

And their marketplace page:  https://github.com/marketplace/zenhub/plan/MDIyOk1hcmtldHBsYWNlTGlzdGluZ1BsYW41OQ==#pricing-and-setup

Would Antonin, Tom, Owen, be ok with us enabling ZenHub and at least giving it a spin?

Thad

https://www.linkedin.com/in/thadguidry/

[Thad Guidry]

Hi Dev Team!

Another gentle reminder.

Anyone against enabling ZenHub to help us organize a bit better? (see below)

If not then I'd like to enable it tomorrow.

And their TOS and Privacy policiy are here for anyone concerned: https://www.zenhub.com/terms-of-service  and https://www.zenhub.com/privacy-policy

Thad

https://www.linkedin.com/in/thadguidry/

[Antonin Delpeuch (lists)]

I am not convinced by ZenHub's architecture, since it relies on browser
extensions to work. This seems brittle and will only benefit those who
install the extension.

I am also reluctant to invest in tools that are GitHub-only, as they
increase the vendor lock-in. Microsoft is capable of making the
platforms it acquires significantly worse (Skype is one example).

Antonin

On 27/08/2020 21:13, Thad Guidry wrote:
> Hi Dev Team!
>
> Another gentle reminder.
> Anyone against enabling ZenHub to help us organize a bit better? (see below)
> If not then I'd like to enable it tomorrow.
> And their TOS and Privacy policiy are here for anyone concerned:
> https://www.zenhub.com/terms-of-service  and
> https://www.zenhub.com/privacy-policy
>
> Thad
> https://www.linkedin.com/in/thadguidry/
>
>
> On Sat, Aug 22, 2020 at 3:40 PM Thad Guidry <thadg...@gmail.com
> <mailto:thadg...@gmail.com>> wrote:
>
> Hi Dev Team!
>
> And I'm still disgusted that GitHub does not have good project
> management features like Jira or Phabricator to see and plan a
> roadmap with dependencies.  But the good news is that GitHub support

> recommends *ZenHub* that does just that and more and supports

> dependencies and it's all right within GitHub's UI.  Azure Boards
> doesn't integrate or sync with GitHub issues, I tried it and also
> saw their roadmap.
>

> I would like us to enable *ZenHub* (free for us since we are open

> source) on the OpenRefine org in GitHub and then we could make those
> dependencies and have real EPICs (instead of the fake ones I've been
> creating), and then everyone can participate with their same GitHub
> login.
> This benefits us in lots of ways.  Unlike Phabricator or Jira, no
> hosting needing, 100% free for us, and no new application to learn,
> it's native inside of GitHub and simply enables extra features.
>
> Take a look:
> https://help.zenhub.com/support/solutions/articles/43000010349-create-github-issue-dependencies
> And their marketplace page: 
> https://github.com/marketplace/zenhub/plan/MDIyOk1hcmtldHBsYWNlTGlzdGluZ1BsYW41OQ==#pricing-and-setup
>
> Would Antonin, Tom, Owen, be ok with us enabling ZenHub and at least
> giving it a spin?
>
> Thad
> https://www.linkedin.com/in/thadguidry/
>

> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com
> <mailto:openrefine-de...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/openrefine-dev/CAChbWaNgXhyxnC%3D_n%2BzfWCZoTipLXhCh5YH1xDqGd7Mr%2B8Pr3A%40mail.gmail.com
> <https://groups.google.com/d/msgid/openrefine-dev/CAChbWaNgXhyxnC%3D_n%2BzfWCZoTipLXhCh5YH1xDqGd7Mr%2B8Pr3A%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Thad Guidry]

OK.

Thad

https://www.linkedin.com/in/thadguidry/

To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/cefc7b8e-e38e-a66f-c602-fe48bac4548a%40antonin.delpeuch.eu.

[Tom Morris]

I've got no problem with Github, but I didn't reply because I didn't see a problem that needed solving. 

What is it that we need to "manage"? If we can describe the problem clearly, then we can evaluate potential solutions.

Tom

[Thad Guidry]

Tom,

Dependent issues being grouped as Epics.

 

Thad

https://www.linkedin.com/in/thadguidry/

--

You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/CAE9vqEHDejKcAC1K0f7s%2BrHA41gV4OT8HNDzwLXjyEYYwfdnuw%40mail.gmail.com.

[Tom Morris]

On Thu, Aug 27, 2020 at 4:08 PM Thad Guidry <thadg...@gmail.com> wrote:

Dependent issues being grouped as Epics.

I don't think I've seen many (any?) epics for OpenRefine, but a simple mechanism that I've used for other projects on Github is to make an "epic" issue with a checklist and a link to each related issue. The checklist mechanism is a little redundant since the links show up as crossed out when the other issues are closed, but can be used for small tasks that don't rate their own issue.

Tom

[Thad Guidry]

Tom,

Which is what I am also used to and was encouraging the team to do and started to introduce for our project.

I am used to very organized projects.  Ours has been much more freeform as we all know.  I was just trying to help bring some structure.

I still think we should try Zenhub.  But I'll leave it up to the devs here.

All the best,

Thad

https://www.linkedin.com/in/thadguidry/

--

You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/CAE9vqEEE2HE16S6k_4FJ_7vxtZWQkY6h1XAEAkTAv%2B-WksDZWA%40mail.gmail.com.

[Thad Guidry]

Also, my other motivation for this was to help address your concerns mentioned before... Roadmap.

Thad

https://www.linkedin.com/in/thadguidry/

[Thad Guidry]

Antonin,

Would you be more comfortable with a Phabricator instance for us?

Thad

https://www.linkedin.com/in/thadguidry/

[Antonin Delpeuch (lists)]

I don't think we should have two ticket systems, so that would mean
migrating our issues to Phabricator, but then I suspect we would not be
able to keep the same automation for linking pull requests to issues,
for instance.

Antonin

> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com

> <mailto:openrefine-de...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openrefine-dev/CAChbWaMVvd7ds%3DKHcvcOYGuHWwHYk10P9ffjbwDvHXyEvG89OA%40mail.gmail.com
> <https://groups.google.com/d/msgid/openrefine-dev/CAChbWaMVvd7ds%3DKHcvcOYGuHWwHYk10P9ffjbwDvHXyEvG89OA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Thad Guidry]

Antonin,

Yes I agree, I don't want to split efforts.

I'll put this conversation on the back log for now.

If or when GitHub improves their Issue and Project management, then we can revisit this.

Thad

https://www.linkedin.com/in/thadguidry/

To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/d2a1a049-7236-f418-1670-cf71b147438a%40antonin.delpeuch.eu.

[Martin Magdinier]

We already have the project feature in Github to group ticket together and track progress: https://github.com/OpenRefine/OpenRefine/projects

Regarding the roadmap aspect, at some point, we floated the idea to draft a big idea to draft a new idea/feature in a hackmd.io document and potentially add the link in a ticket. At the time the rationale was: 

* Draft the idea using Hackmd

* Hackmd track who contributes and allow for comment 

* One issue in github with the link to the document. The issue in Github allows us to find the issue among 

* As we form a consensus in the hackmd doc we can create the related issues in Github and group them in a project 

We would usee hackmd over Google doc to promote open source. 

This approach will also help to bring together the different separate roadmap document we have been working for CS&S and funding opportunities. 

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/CAChbWaOL5Te6GFYsDdVrdGDHfHtBzAmGgzUB92Ay1aDLzVMxuQ%40mail.gmail.com.

[Thad Guidry]

Martin,

I like the approach of forming a collaborative roadmap on a Hackmd.io document, and then using GitHub Projects to give an overview of related issues.

Essentially, we could use 1 GitHub Project = 1 Epic.  But to fulfill a roadmap stage might be many Epics and that is fine and makes sense to me (and how I'd manage it in Jira if we had)

This makes more sense than using the fake Epic Issues I've been creating in GitHub in my opinion.  (I've removed my unnecessary Projects now from our GitHub Projects)

Tom and Antonin,

Would both of you be willing to coordinate some time together towards the beginning of a roadmap on a Hackmd.io document?

I think it should come from both of you involved only first as a first round.  If I get involved initially, it will fall apart because my vision is really forward reaching beyond 2 years, and I've spoken enough about it and documenting my thoughts that I think both of you know my desires well enough.

Thad

https://www.linkedin.com/in/thadguidry/

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/CAO%2BzzMt4dWmLCRHrPatf-GOdv9PM8EkD2WYgPKu5MUZtJSpQtw%40mail.gmail.com.

[Martin Magdinier]

for clarification, the idea is to have one hackmd doc per idea. So they can move forward at their own pace depending on the community interest and involvement. 
 You can group them in their own GitHub project.

Martin

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/CAChbWaO5dES3X-meOx%2Bm0CBkiML08mYyw7DWHP8mp9%2Bw_aUJPg%40mail.gmail.com.

[Tom Morris]

My planning needs are really simple and don't require fancy tooling. A spreadsheet, bulleted lists in email, or Github milestones / project boards would all work.

The important stuff is the content. When I ask "what's targeted for the 3.4 end-game?", instead of crickets, I hear "Nothing, we're all set" or "Two critical data corruption bugs: #42 & #314" and then I can say, "OK, I'll tackle #42." 

Putting some stakes in the ground for dates, like we're going to have a quarterly or annual (or whatever) release cadence, or our next two releases will be 3.5 in October and 3.6 in April, or whatever... Then we know how big the buckets are that we're filling.

OpenRefine 3.5 - Nov 2020

• GSoC & Outreachy projects
• Internationalization / Localization
• Fix Date handling
• Migration support for extension writers + API stability improvements
• jQuery 1.12 + jQuery Migrate
• Performance / scalability low hanging fruit
• ...

OpenRefine 3.6 (3.5.1?) - Mar 2021

• bug fix release
• jQuery 3.5.x + jQuery Migrate
• Promises + removal of non-async AJAX calls
• ...

Other, not release based

• Front end test suite
• Performance test suite
• ...

Two years is a pretty long time in software development. I think if we could map out the next couple of 3.x releases and plan how we evaluate and productize the Spark prototype, we'd be way ahead of the game -- say, 15 months through the end of 2021.

Tom

[Thad Guidry]

Tom,

Those stakes are very useful to have and pretty much what I was hoping to get from either one of you.

So thanks for that.

For users however, this is all technical debt more or less that you have listed.  And doesn't bring much to the table for them other than stability and minor performance bumps in OpenRefine in 3.x  Which is completely fine if the roadmap for the next year through 2021 is essentially resolving all of that technical debt that's been building up over years (and that previous others introduced for us ;-)  )

While the focus on 4.x is eventual productize of the Spark prototype, then I'm fine with that.

I just needed to hear that from either of you and as you say, put some stakes in the ground.

Thad

https://www.linkedin.com/in/thadguidry/

--

You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/CAE9vqEF7T9sJAUyp5KH_mEKUGsJa3-CXsmgbQdoF%3DHB0iGUgqg%40mail.gmail.com.

[Thad Guidry]

Would either of you mind clicking edit and filling in the dates for those milestones and cleaning them up a bit here?  That would help.
https://github.com/openrefine/openrefine/milestones

All of this helps in being able to refer folks to our general roadmap of development effort for the next year.

I see Antonin's document being closer to a 2 year plan.

Thad

https://www.linkedin.com/in/thadguidry/

[Tom Morris]

On Fri, Aug 28, 2020 at 12:31 PM Thad Guidry <thadg...@gmail.com> wrote:

Those stakes are very useful to have and pretty much what I was hoping to get from either one of you.

That's just a top of the head sketch to get conversation rolling (without looking at the backlog). What do others think? Do we want 3 month releases? 6 month releases? Annual? What other important themes, features, or bug fixes should be considered for inclusion?

 

For users however, this is all technical debt more or less that you have listed. 

Technical debt refers to deferred maintenance that hinders development velocity. That doesn't really affect users except in that it delays when they get new features or bug fixes. Long ignored bugs or feature requests are a different thing and provide immediate user benefit when they are fixed or implemented. jQuery (and perhaps async AJAX) are the only real tech debt items. All the localization cleanups are bug fixes or enhancements. Fixing Dates I won't characterize because it'll make me too angry.

 

And doesn't bring much to the table for them other than stability and minor performance bumps in OpenRefine in 3.x 

I'm sure there are other worthwhile things buried in the backlog. Like I said, I didn't review it. I'm open to suggestions of other high priority bugs or feature requests to address.

Although I've done a bunch of work on localization, I think it deserves continued focus with OpenRefine's increasingly diverse user base. Embedded English stop words, US-centric date formatting, etc could all be cleaned up.

Better support for exploratory data analysis with visualizations, statistics, etc would benefit a lot of people. 

People have mentioned wanting to turn the undo history into a properly supported DSL. That would be a significant chunk of work in its own right.

Let's cast a wide(ish) net to collect candidates, then prioritize and prune the list.

Tom

 

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/CAChbWaPFRTDCT%3D9e8%3DFZrnvKLsLp0GNsa-j_8nfqvk8s%2BRt-UA%40mail.gmail.com.

[Thad Guidry]

Good action plan, Tom.

I prefer 6 month release, this aligns better with contractual work, internships, etc.

The 2 general themes that I saw (and recall from David's conversations) were around UI and Performance. (which is why I also had them as GitHub Projects, but now closed them, but still visible with some linked issues)

For suggestions on what to prioritize...

The narrower community of devs and advanced users that participate in our GitHub Issues say its this  (sorted descending by reactions):

https://github.com/OpenRefine/OpenRefine/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc

Where you can kinda see that's pretty much aligns with what Antonin is prioritizing and working down from with his efforts.

But our greater community is looking forward to many of these on our High priority label listing (which includes the often asked "free text search across all Columns"):

https://github.com/OpenRefine/OpenRefine/issues?q=is%3Aopen+is%3Aissue+label%3A%22priority%3A+High%22

Part of my reasoning also for pushing the team on helping prioritize, develop roadmaps, create Epics (or idea docs) and dependencies is so that we can evaluate that against new contractual work and proposals for work by paid and unpaid volunteers and with an eye towards asking for future funding.

Thad

https://www.linkedin.com/in/thadguidry/

[Tom Morris]

In the last 5 months we haven't made much progress on the release planning discussion, but we have almost 90 bug fixes, some dating back to May 2020, so it seems like we should try to get another release out the door sooner rather than later. It seems silly to sit on completed work for a year.

Our last two releases were:

- v3.3 - Jan 2020

- v3.4 - Sep 2020

The earliest that we could probably realistically get a release out the door is April.

I'll start a separate OpenRefine 3.5 planning thread, but wanted to ping this old thread to encourage people to think about timing and content.

Tom

[Thad Guidry]

Gosh, thank you so much Tom for driving forward another release.

Thad

https://www.linkedin.com/in/thadguidry/

https://calendly.com/thadguidry/

--

You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openrefine-dev/CAE9vqEEGN67Np1%2BMxJh%3DPErW5WGHKaDPYg8yNfZmZR8EN0KY8g%40mail.gmail.com.