Apache Foundation vs SF Conservancy?

[Antonin Delpeuch (lists)]

Hi all,

I had a discussion with Giovanni Tummarello and Fabio Tacchelli from
Siren.io some weeks ago and they raised the question of whether
OpenRefine had considered the Apache Software Foundation (ASF) as an
institutional host.

I have to admit I did not even think about it myself when we considered
our options. In my mind Apache projects are very established and mature
- but that sort of expectation could be one good reason to join them (if
possible): we would get a valuable quality seal.

We are currently at a fairly advanced stage into joining the SF
Conservancy (SFC). I still think SFC would be a great match for
OpenRefine. They also host very reputable projects.

I think it would be worth comparing the two options (and other similar
organizations, if you can think of any) to make sure we go down the
right route.

Among the main differences I can see:
- joining SFC does not require transferring intellectual property,
joining ASF does
(https://www.apache.org/foundation/how-it-works.html#incubator)
- SFC can handle funds to pay developers, ASF does not do that
(https://www.apache.org/foundation/how-it-works.html#operation)
- ASF is more famous than SFC. The "Conservancy" name (and the website)
sort of gives the impression to passers-by that they exist to "conserve"
an heritage of legacy projects rather to foster the development of
active ones. (That is a very superficial comment to make, but hey, image
counts too.)

As you also know we are currently investing a lot of effort migrating
out of the "org.json" library for licensing reasons. SFC require that
OpenRefine does not depend on this library before we can join them.
Well, ASF have the exact same requirement so this work needs to be done
no matter which organization we join. (See
https://www.apache.org/legal/resolved.html#json)

AFAICT the main drive to join the SFC is to handle grants such as the
one from the Google News Initiative. It is not clear to me how that
would work in the ASF - we would need another institution to handle this
sort of grant? (Does that mean we can join both the SFC and ASF
simultaneously, given that they seem to do quite different things?)

Let me know what you think!
Antonin

[Martin Magdinier]

Antonin, 

Thanks for the comparison and I agree with your point. On one side, the Apache Software Foundation provides more visibility. On the other hand, the Software Freedom Conservancy has more flexibility.  Note that the ASF can accept donation (see here). However, it is not clear how the funds can be used. 

For the record, transferring intellectual property to the ASF is tricky. Currently, each OpenRefine contributor owns the intellectual property of their code (or the company that paid for the work). It means we will need to track each PR and ask the contributors to transfer their IP ...

My take is we stay with the SFC as we already have been through the selection process. 

I am looking forward to seeing this org.json migration completed! Thanks for your hard work.

--
You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Thad Guidry]

+1 for SFC... we have a financial problem mostly...and SFC fixes that, while ASF doesn't give us guarantees... " If you have a specific target or project that you wish to directly support, please contact us and we will do our best to satisfy your wishes."

Thad

+ThadGuidry

[Giovanni Tummarello]

Hello Thad, hello everyone, sorry for being a bit late to this! :)

I think it would be very exciting for everyone if OpenRefine would become an Apache project

I think its really worth exploring this opportunity because it could make the difference between a popular triving piece of software (with adoption in industry also) and something less visible.

As you know David Huynh, original author together with Mazocchi is in the advisory board at Siren and we discussed this and he also was very excited at this idea (we might hear from him in fact).

I see two points

1) handling funds

2) the IP rights.

I think 1) is solvable, maybe requires setting up a no profit but that's doable, we coulds upport that as Siren

so the bigger point would be 2) ..  openrefine has been Apache for the logest time, from its google contribution to opensource being absolutely clear and unquestionable. I would propose to have a chat with someone senior at Apache and see what they think specifically.

I'd be happy to work on this if you guys are similarly exciting at the pespective (if 1 and 2 can be worked out)

cheers

[Thad Guidry]

Hey Giovanni !

If 1 can be worked out... then yes absolutely !

I spent a week trying to come to grips with Non-Profit setup here in Dallas with a few lawyers and it was slow going and we needed to quickly get a business account to collect funds, so that's why OpenRefine Foundation came into being as a private entity for accounting purposes only.

If Siren can help us with 1 and setting up the Non-Profit and make it seamless for us to be able to collect Private and Public money, AND help with filing every year/quarter to comply with any IRS rules, then that should be all we need.

If Apache can help us with 2 and move into incubator status, then that would be a great start !

Thad

+ThadGuidry

[Antonin Delpeuch (lists)]

In my experience setting up a nonprofit is not that much effort, but
running it on the long term is another matter. It takes a lot of work to
get things right, have a sane gouvernance and transparent operations.
That is why SFC feels like a good fit: this is exactly what they
provide. If joining Apache requires setting up another institution on
the side, it's a bad start: the non-profit we set up on the side will
not benefit from us joining a larger organization.

Also, time-wise, I doubt we can complete an Apache migration in the next
few months.

I think we should stick with the SFC migration for now. We can consider
joining Apache (which is a long term project) once this is complete (and
again, what prevents us from being both in SFC and Apache?)

Antonin

On 11/15/18 2:30 PM, Thad Guidry wrote:
> Hey Giovanni !
>
> If 1 can be worked out... then yes absolutely !
>
> I spent a week trying to come to grips with Non-Profit setup here in
> Dallas with a few lawyers and it was slow going and we needed to quickly
> get a business account to collect funds, so that's why OpenRefine
> Foundation came into being as a private entity for accounting purposes only.
>
> If Siren can help us with 1 and setting up the Non-Profit and make it
> seamless for us to be able to collect Private and Public money, AND help
> with filing every year/quarter to comply with any IRS rules, then that
> should be all we need.
> If Apache can help us with 2 and move into incubator status, then that
> would be a great start !
>
> Thad

> +ThadGuidry <https://www.google.com/+ThadGuidry>

>
>
> On Wed, Nov 14, 2018 at 9:36 PM Giovanni Tummarello
> <g.tumm...@gmail.com <mailto:g.tumm...@gmail.com>> wrote:
>
> Hello Thad, hello everyone, sorry for being a bit late to this! :)
>
> I think it would be very exciting for everyone if OpenRefine would
> become an Apache project
>
> I think its really worth exploring this opportunity because it could
> make the difference between a popular triving piece of software
> (with adoption in industry also) and something less visible.
>
> As you know David Huynh, original author together with Mazocchi is
> in the advisory board at Siren and we discussed this and he also was
> very excited at this idea (we might hear from him in fact).
>
> I see two points
>
> 1) handling funds
> 2) the IP rights.
>
> I think 1) is solvable, maybe requires setting up a no profit but
> that's doable, we coulds upport that as Siren
>
> so the bigger point would be 2) ..  openrefine has been Apache for
> the logest time, from its google contribution to opensource being
> absolutely clear and unquestionable. I would propose to have a chat
> with someone senior at Apache and see what they think specifically.
>
> I'd be happy to work on this if you guys are similarly exciting at
> the pespective (if 1 and 2 can be worked out)
>
> cheers
>

> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com

> <mailto:openrefine-de...@googlegroups.com>.

[Giovanni Tummarello]

Hi Antonin, Thad

the Conservancy seems to be great for the funding aspect, sparing the need for a non profit or delaying it.

In general i do see supporting a no profit possible - i would have to look specifically into the costs and make a case for it. .but hey.. the grants can cover the costs too. 

I propose to go in steps.. first lets find out well about the possibility of doing a step1 and then proceding right away with step2 if that's compatible. 

This would mean engage with Apache and discuss if they see any problem with this. I could initiate that? 

[Thad Guidry]

I am OK with initiating that.  BUT, that means telling the SFC to put the brakes on - so they don't spend unnecessary resources on a dual effort on their side for now.

We will also need to have Martin and Antonin and Jacky agree as well.

Martin ?

Jacky ?

Antonin ?

Thad

+ThadGuidry

--
You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.

[Antonin Delpeuch (lists)]

If it is just about asking ASF what solution they propose to handle
funds, I am not sure we need to stall the SFC application for that?

I would prefer to keep the SFC application going for now (and it's not
like they are actively working on it - they are waiting for us to
migrate out of org.json).

But yes of course I don't have any issue with anyone asking ASF if and
how they handle project funding (that does need to be formulated as an
application to join yet).

Antonin

On 11/16/18 10:23 AM, Thad Guidry wrote:
> I am OK with initiating that.  BUT, that means telling the SFC to put
> the brakes on - so they don't spend unnecessary resources on a dual
> effort on their side for now.
>
> We will also need to have Martin and Antonin and Jacky agree as well.
>
> Martin ?
> Jacky ?
> Antonin ?
>
> Thad

> +ThadGuidry <https://www.google.com/+ThadGuidry>

> <mailto:openrefine-de...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>

> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com

> <mailto:openrefine-de...@googlegroups.com>.

[Thad Guidry]

OK, good point Antonin, SFC is waiting on us for org.json cutting.

So, no need to stop SFC process.  Good.

Giovanni - you are welcome to assist with asking ASF on project funding options and how they work.

Thad

https://www.linkedin.com/in/thadguidry/

To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.

[Giovanni Tummarello]

Great, will engage and let you know

[Tom Morris]

We've used Software Freedom Conservancy (SFC) for ArgoUML and CWL with good success, so I have no complaints about them. Comparing them to the Apache Foundation, though, is like comparing apples and oranges. Joining Apache would be a *much* bigger lift with IP assignment, adopting their governance rules, and a whole stack of other stuff that is not involved in joining the SFC.

And, while the Apache brand is strong, it doesn't confer automatic visibility or success. Putting "Apache" in front of the name doesn't automatically put you on the same footing as the Apache Web Server. For example, they've got two competing web performance tools, Apache Bench and Apache jMeter. Have you heard of either of them? Both? Probably only if you've been involved in web performance work at some point. I'm doubtful that the Apache brand would be a significant marketing benefit to OpenRefine.

I'm not sure I understand Siren's involvement (are they taking over support of the UCG RDF extension?), but I'd certainly be interested in hearing David Huynh's opinion (directly, not second hand).

Tom

[Antonin Delpeuch (lists)]

Thanks very much Tom for your comments. You are making excellent points.
In my opinion that strengthens SFC's case, but I would also be very
interested in hearing David's opinion.

Antonin

> <https://apache.org/foundation/contributing.html#Fundraising>

> and we will do our best to satisfy your wishes."
>
> Thad

> +ThadGuidry <https://www.google.com/+ThadGuidry>

>
>
> On Wed, Nov 7, 2018 at 8:43 PM Martin Magdinier
> <martin.m...@gmail.com> wrote:
>
>
> Antonin, 
>
> Thanks for the comparison and I agree with your point. On
> one side, the Apache Software Foundation provides more
> visibility. On the other hand, the Software Freedom
> Conservancy has more flexibility.  Note that the ASF can
> accept donation (see here

> <https://apache.org/foundation/faq.html#how>). However, it

> <mailto:openrefine-de...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com

> <mailto:openrefine-de...@googlegroups.com>.

[Thad Guidry]

I don't think its about Apache branding at all (our users could care less).  And I personally only care about our users.  I enjoy the Apache Way.

In fact, we are very much already operating the Apache Way.  Read a few paragraphs here: http://www.apache.org/foundation/how-it-works.html#management

Where really the biggest change for us would be to nominate this individual:

Project Management Committees

PMCs vote on new committers and PMC members for their project, and set per-project policies as well as formally voting on software product releases. Read about PMCs. PMCs report quarterly directly to the board, not to the President.

Currently, we have a PMC, set as myself as entrusted by our committers and David Huynh.  And I thank him and everyone for trusting me in that capacity currently.

For those that are worried that joining the ASF would put constraints on how we currently do things...no, not much at all, because...

"Since the appointed Project Management Committees have the power to create their own self-governing rules, there is no single vision on how PMCs should run a project and the communities they host."

Tom - I think I would value that "other stuff" as a benefit to joining ASF.

    We would enjoy their governance rules, which are pretty much in line with what we do now, but we would finally have something written.  SFC enjoys no such benefit.

    We would enjoy Events and Mentoring.  SFC enjoys no such benefit.

    We would enjoy their Code of Conduct and Etiquette guidelines.  SFC enjoys no such benefit.

Our financials would certainly be different, but better in many ways.

Our users, if they wanted to donate directly to OpenRefine usage, would have to contact fundr...@apache.org ...

  but outside of that would have a donation page to go to directly via here: http://apache.org/foundation/contributing.html

Yes, ASF absorbs 15% via Targeted Sponsorship, which is more than SFC's 10%... in other words, if Google or some entity wanted to help OpenRefine again, it would be done under the ASF's Targeted Sponsorship program as noted here: http://apache.org/foundation/docs/Targeted-Sponsorship-Policy.pdf

But with ASF we wouldn't have to worry about any of the financials with ASF.  (Well,  our PMC would need to review Targeted Sponsorships that come in)

With SFC, we would still have to worry about donation handling, sponsorship handling (SFC admitted that already to us on the call), etc.  They just help as a non-profit placeholder and really nothing more.

Thad

https://www.linkedin.com/in/thadguidry/

[Antonin Delpeuch (lists)]

On 11/19/18 10:30 AM, Thad Guidry wrote:
> Currently, we have a PMC, set as myself as entrusted by our committers
> and David Huynh.  And I thank him and everyone for trusting me in that
> capacity currently.

For the record can you give a link to where this was discussed?

Antonin

[Thad Guidry]

It was a meritocracy arrangement. Some were private emails and chat, were some included you as well (the business account handling, and Google News initiation).

I would need to get approval from Stefano and David and Tom and Martin to allow me to share those with you, since there was personal info contained in them.

I have them as full PDF's (just printed the whole threads) and could share with you privately once they said yes.

For Reference the email subjects were:

"M.I.A." - Fri, Feb 1, 2013 at 1:06 PM - initiated by David Huynh

"Admin access for OpenRefine-dev mailing list" -  Sun, Nov 26, 2017 at 3:53 PM - initiated by Thad Guidry

Public Mailing list:

https://groups.google.com/d/msg/openrefine-dev/T_oamGet9jA/qzt_Cz-EBQAJ

https://groups.google.com/d/msg/openrefine-dev/7zq0InpZFrM/iVNumom-Q4AJ

https://groups.google.com/d/msg/openrefine-dev/xfO-BpfObRg/hwnmmy5jWWsJ

https://groups.google.com/d/msg/openrefine-dev/0lkkYwrP2B8/MBVlFb4ErYQJ

Thad

https://www.linkedin.com/in/thadguidry/

[Thad Guidry]

But Antonin... I don't think the arrangement matters...let me explain...

All of us here have a responsibility to make wise choices for OpenRefine's long-term health.  This isn't about me.  I am just one voice.  It is all of our voices and opinions that are equally valued here.  Government by merit. "meritocracy"

Those that have merit, i.e. prove valuable to the project, You, Me, Owen, Jacky, Martin.

The next PMC could be you, or it could be you right now in reality.

Honestly, this doesn't matter, as long as we decide together as this small little group we have (ever so small!) on what is best for OpenRefine and its long term future.

Personally, I would have liked to have seen the segway handled earlier in OpenRefine's timeline, but everyone (Martin, Myself, Tom, David, Stefano) was so busy between 2013-2016 quite honestly.

And I'm really glad this current discussion is happening in the public ! :)  Martin and I didn't quite like having those private conversations, but I understand why David and Stefano did so. (avoiding community backlash if it might happen, but were such a small community to begin with it probably would have been nil in my opinion)

Thad

https://www.linkedin.com/in/thadguidry/

[Ettore RIZZA]

Those that have merit, i.e. prove valuable to the project, You, Me, Owen, Jacky, Martin. 

Get a room! :D

Ettore Rizza

[Antonin Delpeuch (lists)]

Sure, you are doing great work and I trust you for that!
It's just that I had never heard of the PMC term in the context of
OpenRefine so I was curious to see why you were suddenly asserting that
"Currently, we have a PMC, set as [yourself]"… it seems a little bold!

Migrating to organizations like SFC or Apache requires taking project
gouvernance seriously I think, so let's be precise about these things :)

Antonin

> <mailto:openrefine-dev%2Bunsu...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>

> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com

> <mailto:openrefine-de...@googlegroups.com>.

[Antonin Delpeuch (lists)]

Ettore I am afraid our Project Management Committee has declared you had
no merit! Oops!

More seriously, yes you are doing great work and you should totally have
your say in these things!

Antonin

On 11/19/18 11:53 AM, Ettore RIZZA wrote:
> Those that have merit, i.e. prove valuable to the project, You, Me,
> Owen, Jacky, Martin. 
>
>
> Get a room! :D
>
> Ettore Rizza
>
>
> Le lun. 19 nov. 2018 à 12:44, Thad Guidry <thadg...@gmail.com

> <mailto:thadg...@gmail.com>> a écrit :

> <li...@antonin.delpeuch.eu <mailto:li...@antonin.delpeuch.eu>>

> wrote:
>
> On 11/19/18 10:30 AM, Thad Guidry wrote:
> > Currently, we have a PMC, set as myself as entrusted by
> our committers
> > and David Huynh.  And I thank him and everyone for
> trusting me in that
> > capacity currently.
>
> For the record can you give a link to where this was discussed?
>
> Antonin
>
> --
> You received this message because you are subscribed to the
> Google Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails
> from it, send an email to
> openrefine-de...@googlegroups.com

> <mailto:openrefine-dev%2Bunsu...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to openrefine-de...@googlegroups.com

> <mailto:openrefine-de...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com

> <mailto:openrefine-de...@googlegroups.com>.

[Thad Guidry]

Ettore,

We would not be a community without you taking your time (unpaid still !) in answering hundreds of questions on our mailing list and Stackoverflow.

We are continually indebted to you my friend.

Did you ever get a sticker at least sent to you ???

Thad

https://www.linkedin.com/in/thadguidry/

[Ettore RIZZA]

I already have it. ;)

Thank you Antonin/Thad, but I was joking of course. We are on the mailing list "developers" after all, and the list is long (thankfully!) of all those who contribute to enrich or popularize OpenRefine.

Having said that, I do not know at all what role all these people could play in the structures you are discussing. PMC voters?

Best,

Ettore

[Thad Guidry]

Ettore,

Their role could be any number of these... User, Developer, Committer, PMC member, but most fall under Developer.

http://www.apache.org/foundation/how-it-works.html#roles

You would be 3 in fact. User, Developer, PMC member

Thad

https://www.linkedin.com/in/thadguidry/

[Giovanni Tummarello]

Hi Thad,

it seems you've found the info we required. I'd say this sounds really good.  What could be a next step? 

I have asked David to share his opinion directly,  here so lets see if that happens, hoping so :) 

About Siren (hi Tom!), we envision investing development resources in the project in 2019. An Apache project would be ideal.

 I started originally the Apache top level any23 project with a donation of code from my group at NUIG (any23 was part of the Sindice engine and Siren official name is Sindice ltd so see the connection :) ) and have been really impressed on how a community formed around it and the result was of benefit to many.

Gio

[Antonin Delpeuch (lists)]

Hang on, why is that the info we required? Quoting the document on
targeted sponsorships:

> Examples of Targeted Sponsorships that are Highly Unlikely:
> - Developing code
> ...

So - I still don't see how ASF could be the right choice to get
subsequent grants from Siren or any other sponsor?

Antonin

On 11/19/18 3:35 PM, Giovanni Tummarello wrote:
> Hi Thad,
>
> it seems you've found the info we required. I'd say this sounds really
> good.  What could be a next step? 
>
> I have asked David to share his opinion directly,  here so lets see if
> that happens, hoping so :) 
>
> About Siren (hi Tom!), we envision investing development resources in
> the project in 2019. An Apache project would be ideal.
>
>  I started originally the Apache top level any23 project with a donation
> of code from my group at NUIG (any23 was part of the Sindice engine and
> Siren official name is Sindice ltd so see the connection :) ) and have
> been really impressed on how a community formed around it and the result
> was of benefit to many.
>
> Gio
>
>
> On Monday, November 19, 2018 at 10:30:23 AM UTC, Thad Guidry wrote:
>
> I don't think its about Apache branding at all (our users could care
> less).  And I personally only care about our users.  I enjoy the
> Apache Way.
>
> In fact, we are very much already operating the Apache Way.  Read a
> few paragraphs here:
> http://www.apache.org/foundation/how-it-works.html#management
> <http://www.apache.org/foundation/how-it-works.html#management>
> Where really the biggest change for us would be to nominate this
> individual:
>

> *Project Management Committees
> <https://www.apache.org/foundation/governance/pmcs.html>*

>
> PMCs vote on new committers and PMC members for their project, and
> set per-project policies as well as formally voting on software
> product releases. Read about PMCs

> <https://www.apache.org/foundation/governance/pmcs.html>. PMCs

> report quarterly directly to the board, not to the President.
>
> Currently, we have a PMC, set as myself as entrusted by our
> committers and David Huynh.  And I thank him and everyone for
> trusting me in that capacity currently.
>
> For those that are worried that joining the ASF would put
> constraints on how we currently do things...no, not much at all,
> because...
> "Since the appointed Project Management Committees have the power to
> create their own self-governing rules, there is no single vision on
> how PMCs should run a project and the communities they host."
>
> Tom - I think I would value that "other stuff" as a benefit to
> joining ASF.
>     We would enjoy their governance rules, which are pretty much in
> line with what we do now, but we would finally have something
> written.  SFC enjoys no such benefit.
>     We would enjoy Events and Mentoring.  SFC enjoys no such benefit.
>     We would enjoy their Code of Conduct and Etiquette guidelines. 
> SFC enjoys no such benefit.
>
> Our financials would certainly be different, but better in many ways.
> Our users, if they wanted to donate directly to OpenRefine usage,

> would have to contact /fundr...@apache.org <javascript:> /...

>   but outside of that would have a donation page to go to directly
> via here: http://apache.org/foundation/contributing.html
> <http://apache.org/foundation/contributing.html>
>
> Yes, ASF absorbs 15% via Targeted Sponsorship, which is more than
> SFC's 10%... in other words, if Google or some entity wanted to help
> OpenRefine again, it would be done under the ASF's Targeted
> Sponsorship program as noted here:
> http://apache.org/foundation/docs/Targeted-Sponsorship-Policy.pdf

> <http://www.google.com/url?q=http%3A%2F%2Fapache.org%2Ffoundation%2Fdocs%2FTargeted-Sponsorship-Policy.pdf&sa=D&sntz=1&usg=AFQjCNFDfej5nDQATBpyMbebjAkd3AKvuw>

> But with ASF we wouldn't have to worry about any of the financials
> with ASF.  (Well,  our PMC would need to review Targeted
> Sponsorships that come in)
> With SFC, we would still have to worry about donation handling,
> sponsorship handling (SFC admitted that already to us on the call),
> etc.  They just help as a non-profit placeholder and really nothing
> more.
>
> Thad
> https://www.linkedin.com/in/thadguidry/

> <https://www.linkedin.com/in/thadguidry/>

[Martin Magdinier]

Giovanni,

I agree with Tom, Thad, and Antonin points listed previously and my vote remains with the SFC. 

Can you explain how the ASF will be better than the SFC for Siren to contribute to the project? 

Thad, 

SFC offers similar services to the ASF :

• Mentoring and events organization services. See https://sfconservancy.org/projects/services/
• The SFC will ask us a formal governance process, see the section How are “project leaders” defined with respect to Conservancy?
 of their application page: https://sfconservancy.org/projects/apply/

Thad, Antonin, Ettore,

The PMC includes all the current committers on the OpenRefine project as listed here: https://github.com/orgs/OpenRefine/people - Ettore you are already a committer :)  You can see our Governance model here: https://github.com/OpenRefine/OpenRefine/blob/master/GOVERNANCE.md. I submitted a change to our Governance page to clarify this section. 

The PMC voted on only one issue in the last 12 months to decide if we are going with the SFC or setting up our own non-profit. The vote was to go with the SFC.

--

Martin Magdinier

To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.

[David Huynh]

Hi all,

Apologies in advance for not being very helpful here :)

Gio told me about the idea of OpenRefine joining Apache a few weeks ago, and I said it sounded great to me. I wasn't aware of SF Conservancy, having left the open source scene years ago. I really don't have an opinion about Apache vs. SFC.

Between Stefano and me, he's always the one much more attuned to open source community matters. I focused primarily more on the users and the conceptualization of the software. He was on the board at Apache years ago, and could provide a grounded opinion.

Re: "PMC" -- I'm not aware I'm on it :) For the last several years I have not found any free time to participate in this community, nor to keep up to date, so from my perspective I have forfeited my claim to any voting right. You're the ones doing all the hard work.

That being said, I wonder if this discussion should be preceded with or grounded on a shared vision discussion: where do you collectively want the project to be, say, 5 years from now? And then, given that shared vision, how do you want some appropriate combination of { SFC and/or Apache, Google and/or Siren } to help it get there?

Thanks,

David

[Martin Magdinier]

David, 

Our vision and roadmap are summarized in: 

• OpenRefine Phase 1 enhancement - for the Google News Grants
  
• OpenRefine Phase 2 enhancements - for the Google News Grants
  
• Github projects.

Last winter (Dec '17 - Jan '18), we started to update the OpenRefine Governance Model and prepare a Governance document for the foundation. Please read those two documents as "draft for discussion." We draft them during our application for the SFC. Nothing there is set in stone. We were just bouncing idea between us and understanding how open source governance works. 

We will start to revisit the governance documents once we resume our application with SFC (i.e., once we clear this ORG.JSON to Jackson transition). I'll be happy to resume those discussions any time. 

What type of support Siren can provide? 

[David Huynh]

Thanks, Martin! Those documents are well thought out!

For more product ideas to bounce around: in the past, I've had discussions with various folks on the following topics, which could make bits and pieces of a longer-term product vision.

1. Charting / visualization while cleaning. Plotting things on maps seems like a common task.
   
2. Reporting / summarization / pivot table creation.
3. Text analysis, such as by integrating the Stanford NLP parser and exposing its parse trees to facet expressions, e.g., creating a facet that lists all the subjects of the sentences in a particular column. (NLP parsing is slow, so probably the first step is to create another column storing the parse trees, which requires extending the cell value type.)
4. Graph-based modeling and set-to-set browsing (pivoting), a la Freebase Parallax: https://vimeo.com/1513562. Siren is actually based on this idea, and could help bring it to OpenRefine. It'd be quite powerful to be able to mix interactive (tabular) data cleaning with on-the-fly graph-based data modeling and browsing, especially if the data (over more than one Refine project) is naturally a knowledge graph.

Cheers,

David

[Tom Morris]

Does the OpenRefine Foundation have non-profit status? What class? One of the big things that joining a larger organization like the SFC brings to the table is to avoid the heavy lifting of having to incorporate. If that's been done already, it may lessen the need for joining.

I don't think the SFC has strong feelings about the form of the governance model other than to be sure that one is in place and they have clear direction regarding who has authority to make decisions. The current documents seem like a good starting point, but it doesn't seem like they received much review and feedback, so it'd be great to pick that back up and drive it to conclusion.

On the product feature front, I think the most important thing is refactoring the processing model to work at larger scales. This is described variously a "separation of front/back end," "apache beam," etc but is actually quite a bit more involved than just tweaking and documenting the existing APIs. There's a fairly fundamental assumption built-in that you can work on all the data at once which breaks at scale. It'd be a challenge, but it'd awesome to be able to have the model be general enough to support Google BigQuery, Vertica column stores, or other types of backends. On a similar front, bridging the interactive analysis/development and automated rerunning of tasks. This is more than just a straight replay, because the runs need to be parameterized in ways not currently supported by the interactive model.

On the UX front, more automation and suggestions for users á la the Stanford DataWrangler work would help get the users out of the low level details and focusing on higher level objectives. Spark lines, quick summaries, etc as mentioned by David's #1 & #2 are part of this.

Tom

[Thad Guidry]

Tom,

No, OpenRefine Foundation is just a private entity in my name (with Martin and myself controlling funds completely in the open), it was quickly setup with community feedback and a quick vote, so that we could collect funds at that time from Google News.

This is why we are looking to attaining non-profit status from SFC or Apache.

(I tried to get non-profit setup initially but it took longer than 1 week and we needed to move fast, because Google News also didn't want to see the funds diverted and wanted to help us.)

You've been away, but YES, that's the kind of model and ideas that I have.  Martin didn't say this, but I was the one who produced the Phase 1 and 2 documents primarily... basically taking David and myself's ideas from long ago and going forward with them.

Correct, we already know and plan to have 2 modes of operation.  Batch and Interactive... knowing that you can only do somethings in one mode and not the other...and that's fine, since other Enterprise tools I use also work exactly the same way.

(A lot of OpenRefine's transforms and operations can be executed via Beam's ptransforms https://github.com/apache/beam#the-beam-model , and some cannot or don't lend themselves well to that, but that's fine)

For automation/suggestions, a bunch is planned, many of them in our heads and talked about sometimes, but not all of them written down until we can separate front/back first so that we can have a richer UI/panels to allow all of that to happen.

OpenRefine is in very good hands.

Time, commits, and a sustainable financial flow (SFC or Apache) are what we need.

Thad

https://www.linkedin.com/in/thadguidry/

[Thad Guidry]

Tom / David,

Here's the other doc that Flavio and I coordinated on regarding Apache Beam experiments , very impromptu and hackish just to capture bits of info ...

https://github.com/OpenRefine/OpenRefine/issues/1468

Flavio's Beam branch is here: https://github.com/OpenRefine/OpenRefine/tree/beamusupscotty

You probably didn't notice it, but we drive the Project using Github's new Project management tooling... each bolded section on the left is hoverable/clickable , so take a look for even more details not captured on enhancement docs ...

https://github.com/OpenRefine/OpenRefine/projects

The Enhancement docs are just some formalization's of what is under Projects to give to Google News or other sponsors and the folks signing big checks ;-)

Thad

https://www.linkedin.com/in/thadguidry/

[Martin Magdinier]

Hello everyone. 

We are accepted in the Software Freedom Conservancy! 

They are asking for us to identify a project leadership committee. The leadership committee will be responsible to sign the agreement between Conservancy and OpenRefine. I suggest we either:

1) Have enough volunteer from our PMC group (identified here)

2) Agree on a governance model for the foundation and elect a board of directors. A draft of the governance rules is available for discussion.

Any preferences? Should we vote on this item?

Martin

[Thad Guidry]

I would like to volunteer to be on the committee.

Can we get their their help on the governance ?

Thad

https://www.linkedin.com/in/thadguidry/

[Antonin Delpeuch (lists)]

I think it would make sense to just take people from the PMC. (if not
all the PMC?) I expect it is possible to update the project leadership
committee later on as commitments evolve?

My understanding is that we will not need the foundation anymore once we
have joined SFC so I am not sure about creating more gouvernance
structure there?

Antonin

On 12/8/18 6:28 AM, Thad Guidry wrote:
> I would like to volunteer to be on the committee.
>
> Can we get their their help on the governance ?
>
> Thad
> https://www.linkedin.com/in/thadguidry/
>
>
> On Fri, Dec 7, 2018 at 3:21 PM Martin Magdinier
> <martin.m...@gmail.com <mailto:martin.m...@gmail.com>> wrote:
>
> Hello everyone. 
>
> We are accepted in the Software Freedom Conservancy! 
>
> They are asking for us to identify a project leadership committee.
> The leadership committee will be responsible to sign the agreement
> between Conservancy and OpenRefine. I suggest we either:
> 1) Have enough volunteer from our PMC group (identified here

> <https://github.com/orgs/OpenRefine/people>)

> 2) Agree on a governance model for the foundation and elect a board
> of directors. A draft of the governance rules

> <https://docs.google.com/document/d/16KntPXkyZfGBwh8moDAzcJ5ZDrtgEKRE2MdLo8lgRog/edit#>

> is available for discussion.
>
> Any preferences? Should we vote on this item?
>
> Martin
>
> Le sam. 24 nov. 2018 à 17:52, Thad Guidry <thadg...@gmail.com

> <mailto:thadg...@gmail.com>> a écrit :

> <http://vis.stanford.edu/wrangler/> work would help get the

> users out of the low level details and focusing on higher
> level objectives. Spark lines, quick summaries, etc as
> mentioned by David's #1 & #2 are part of this.
>
> Tom
>
> --
> You received this message because you are subscribed to the
> Google Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to openrefine-de...@googlegroups.com

> <mailto:openrefine-de...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to openrefine-de...@googlegroups.com

> <mailto:openrefine-de...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine-de...@googlegroups.com

> <mailto:openrefine-de...@googlegroups.com>.

[Martin Magdinier]

I am happy to go with the list from PMC and have them make things evolve. However, I want to make sure we do not volunteer people without their clear approval first. 

I think so far we have the following list of opt-in:

* Thad

* Antonin

* Martin

Who else?

Martin

To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.

[Giovanni Tummarello]

Hi Martin

What type of support Siren can provide? 

We are going to be working on Openrefine likely comes new year. 

We have specifically one developer who is particularly skilled in the codebase and we expect some quick wins with respect to scalability. 

Happy to discuss technically once we're there.

At project level, i feel like restating that looking into joining the Apache foundation would be a really valuable goal. 

I hope the board of directors would see that as valuable too and wont forget this.

happy to see the resumed activity

 

[Martin Magdinier]

Giovanni,  this is excellent news. We are looking forward to your team contribution. 

I copied Wes from Unifyd Insights to this email. They are also doing some work to scale OpenRefine. I know they have some plan to share their work with the open source community. 

Currently, we plan to move forward with the Free Software Conservancy. 

Giovanni, Wes, if there is anything we can do at the governance level to ease contribution from corporations, please let us know. 

Martin. 

--

[Thad Guidry]

Giovanni,

Glad to hear you can contribute with one developer to help us next year.

Curious however, what benefit does Siren get out of this ?

I imagine that since the Siren platform relies on clean-ish data, that you sometimes use or looking to use OpenRefine more for cleaning data prior to ingestion into Siren Federate backend ES ?

Thad

https://www.linkedin.com/in/thadguidry/

--

[Giovanni Tummarello]

Hi Thad correct.

We can currently ingest JDBC and Excel/CSV, with data transformations and service enrichment which is all great, but integrating openrefine would provide and extra/useful way to ingest/process data in a self serving manner for analysts. 

While far from today state of the art with a bit of patience it can be extremely useful as we know. For more demanding tasks people would still have the option to purchase commercial alternatives.

Gio

[Thad Guidry]

Giovanni,

You mean you don't like Google Cloud Dataprep ? ;-)

Thad

https://www.linkedin.com/in/thadguidry/

[Giovanni Tummarello]

I dont think Trifacta writes to Elasticsearch (Certainly the google cloud version doesnt) but other than that its an excellent product. 

Makes sense however to provide something out of the box - also given we'll have a free version soon and we'd like it to be as useful as possible :)  

--
You received this message because you are subscribed to a topic in the Google Groups "OpenRefine Development" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openrefine-dev/Csk4QrKrHiE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openrefine-de...@googlegroups.com.

[Antonin Delpeuch (lists)]

Hi Giovanni,

Can you explain again why you would like to see OpenRefine join ASF
rather than SFC? Concretely, what is the benefit for Siren?

All the best,
Antonin

[Tom Morris]

I would be willing to help out.

To address Antonin's earlier comment, I think a clear governance structure is still valuable, even in the context of the SFC. As a matter of fact, in my experience, they insist on it. It can be super lightweight ("whatever Bill says is the rule"), but they like to have it clear and formalized, so they know who's in charge.

As far as ASF goes, I thought that was already discussed and decided. It wouldn't make sense to join both SFC & ASF, so one or the other should be chosen (and I thought it was SFC).  I too would be curious learn what motivates this recurring push towards ASF.

Tom

[Antonin Delpeuch (lists)]

On 12/11/18 7:33 AM, Tom Morris wrote:
> I would be willing to help out.

Yay!

>
> To address Antonin's earlier comment, I think a clear governance
> structure is still valuable, even in the context of the SFC. As a matter
> of fact, in my experience, they insist on it. It can be super
> lightweight ("whatever Bill says is the rule"), but they like to have it
> clear and formalized, so they know who's in charge.

Yes of course, we should absolutely do that within SFC. I just don't see
the point of doing that within the context of what we call "the
foundation", which is the current entity we use to hold the funds
(before the migration to SFC).

>
> As far as ASF goes, I thought that was already discussed and decided. It
> wouldn't make sense to join both SFC & ASF, so one or the other should
> be chosen (and I thought it was SFC).  I too would be curious learn what
> motivates this recurring push towards ASF.

Me too! As far as I know Siren finds ASF better but I still haven't
fully understood why.

Antonin

>
> Tom
>
> On Mon, Dec 10, 2018 at 3:12 PM Martin Magdinier
> <martin.m...@gmail.com <mailto:martin.m...@gmail.com>> wrote:
>
> I am happy to go with the list from PMC and have them make things
> evolve. However, I want to make sure we do not volunteer people
> without their clear approval first. 
> I think so far we have the following list of opt-in:
> * Thad
> * Antonin
> * Martin
>
> Who else?
> Martin
>
> Le ven. 7 déc. 2018 à 19:09, Antonin Delpeuch (lists)

> <li...@antonin.delpeuch.eu <mailto:li...@antonin.delpeuch.eu>> a écrit :

>
> I think it would make sense to just take people from the PMC.
> (if not
> all the PMC?) I expect it is possible to update the project
> leadership
> committee later on as commitments evolve?
>
> My understanding is that we will not need the foundation anymore
> once we
> have joined SFC so I am not sure about creating more gouvernance
> structure there?
>
> Antonin
>
> On 12/8/18 6:28 AM, Thad Guidry wrote:
> > I would like to volunteer to be on the committee.
> >
> > Can we get their their help on the governance ?
> >
> > Thad
> > https://www.linkedin.com/in/thadguidry/
> >
> >
> > On Fri, Dec 7, 2018 at 3:21 PM Martin Magdinier
> > <martin.m...@gmail.com
> <mailto:martin.m...@gmail.com>

> <mailto:martin.m...@gmail.com

> <mailto:martin.m...@gmail.com>>> wrote:
> >
> >     Hello everyone. 
> >
> >     We are accepted in the Software Freedom Conservancy! 
> >
> >     They are asking for us to identify a project leadership
> committee.
> >     The leadership committee will be responsible to sign the
> agreement
> >     between Conservancy and OpenRefine. I suggest we either:
> >     1) Have enough volunteer from our PMC group (identified here
> >     <https://github.com/orgs/OpenRefine/people>)
> >     2) Agree on a governance model for the foundation and
> elect a board
> >     of directors. A draft of the governance rules
> >   
>  <https://docs.google.com/document/d/16KntPXkyZfGBwh8moDAzcJ5ZDrtgEKRE2MdLo8lgRog/edit#>
> >     is available for discussion.
> >
> >     Any preferences? Should we vote on this item?
> >
> >     Martin
> >
> >     Le sam. 24 nov. 2018 à 17:52, Thad Guidry
> <thadg...@gmail.com <mailto:thadg...@gmail.com>

> >     <mailto:thadg...@gmail.com

> >         <mailto:tfmo...@gmail.com

> <mailto:openrefine-dev%2Bunsu...@googlegroups.com>
> >         <mailto:openrefine-de...@googlegroups.com
> <mailto:openrefine-dev%2Bunsu...@googlegroups.com>>.

> >         For more options, visit
> https://groups.google.com/d/optout.
> >
> >     --
> >     You received this message because you are subscribed to
> the Google
> >     Groups "OpenRefine Development" group.
> >     To unsubscribe from this group and stop receiving emails
> from it,
> >     send an email to
> openrefine-de...@googlegroups.com

> <mailto:openrefine-dev%2Bunsu...@googlegroups.com>
> >     <mailto:openrefine-de...@googlegroups.com
> <mailto:openrefine-dev%2Bunsu...@googlegroups.com>>.

> >     For more options, visit https://groups.google.com/d/optout.
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "OpenRefine Development" group.
> > To unsubscribe from this group and stop receiving emails from
> it, send
> > an email to openrefine-de...@googlegroups.com

> <mailto:openrefine-dev%2Bunsu...@googlegroups.com>
> > <mailto:openrefine-de...@googlegroups.com
> <mailto:openrefine-dev%2Bunsu...@googlegroups.com>>.

> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the
> Google Groups "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from
> it, send an email to openrefine-de...@googlegroups.com

> <mailto:openrefine-dev%2Bunsu...@googlegroups.com>.

[Thad Guidry]

Tom,

Actually, "a governance document" is not needed for joining ASF in the sense that as long as a project has clear documentation somewhere than it is deemed as a "mature Apache project" (CSxx and COxx guidelines along with ALL others - no partial compliance allowed as noted in the Apache Project Maturity Model and ASF has added Footnotes for clarity now)

Over the last 15 years, ASF has loosened restrictions and allowed Projects to run themselves as they see fit, as long as they continue to adhere to the Apache Project Maturity Model as much as possible.

Thad

https://www.linkedin.com/in/thadguidry/

[Giovanni Tummarello]

Hi guys,

I find it a bit surreal to have to explain the difference in terms of perception from an Apache top level project to a SFC,

The world of IT is both powered and contributes to Apache projects. (Active) apache projects are synonym of visibility, trust, governance, high profile. Founder and committers to Apache projects write them in their CVs.

Company announce contributions to apache projects https://www.zdnet.com/article/open-source-gandiva-project-wants-to-unblock-analytics/ large corporation donate  high profile projects as Apache. 

I would say the above should be clear and not require an explanation.

To say that "its the same" .. "it has been decided" seems to me statements contrarian in nature and as such.. will leave them at that.

On my side I think if you guys dont consider applying for Apache a great opportunity will be lost as apache doesn't have a data prep project and this would be a great fit.

On our side we will focus on technical contributions hoping that at least technically we'll be able to find an agreement.

cheers

[Antonin Delpeuch (lists)]

Hi Giovanni,

Thanks for the clarification. So, Siren would prefer to be associated
with an Apache project because this brand would give more credibility to
the tool. I am sure you understand that we, as current contributors of
the project, have other requirements than just branding.

I don't think anybody said "it's the same" (on the contrary, SFC and ASF
operate in quite different ways).

Maybe we can reconsider ASF in a few years if we manage to grow our
contributor base and diversify our funding sources, to the point that we
would run our own non-profit.

I am sure your pull requests will be welcome in any case :)

Cheers,
Antonin

[Giovanni Tummarello]

No, not related  with what Siren would like to be associated, just for clarity.

I simply cared about the uncomparable better way for the tool to foster contributions, attention, users and what not.  But maybe the thing about the donation was a bigger problem that i thought anyway, onward and upward, no worries!

> <mailto:openrefine-dev+unsub...@googlegroups.com>.