Scraping the civic hacking community on GitHub

[Stefan Baack]

Hi everyone,

I’m a PhD student from the Netherlands and I’m currently working on a research project about civic hacking for which I interviewed some members of mySociety — thanks again to everybody who took the time to talk with me!

In these interviews, I asked them to draw a map of all the different organizations or groups that are relevant for mySociety. These maps are very pretty interesting, and they inspired me to do a little experiment: compiling a list of civic tech organizations and scrape some information about them on GitHub. I drafted a blog post about my findings and I would love to hear your thoughts about it! You can find the drafted article here and the data from GitHub here if you’d like to take a closer look.

Best,
Stefan

​

[Zarino Zappia]

Interesting read!

Some clear parallels with my own MSc thesis from 2011, looking at participation in open data development (much of which is what we’d now call civic tech). My data was more limited, but even then, as with your research, certain names would popped up as intermediaries linking disparate communities. mySociety’s own Matthew Somerville comes to mind, or Friedrich Lindenberg at the OKF.

The link if you’re interested: http://zarino.co.uk/post/thesis

As you note, relying on the social features of Github is a poor proxy for actual community ties. I wonder whether you might get a more realistic picture by looking at contributors to repos? So the edges in your network diagram would, essentially, be shared github repositories. It would be less a network of “who’s following who”, and more “who’s actively working with who”.

Also, top designer tip: the data you’re trying to visualise on the Google map is much better suited to a chloropleth map, where you can shade countries/states based on volume of people, rather than using map markers which are prone to overlapping and which make it hard to visually compare areas.

Zarino Zappia

Designer, mySociety

@zarino

--
You received this message because you are subscribed to the Google Groups "mysociety-community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mysociety-commu...@mysociety.org.
To post to this group, send email to mysociety...@mysociety.org.
Visit this group at http://groups.google.com/a/mysociety.org/group/mysociety-community/.
To view this discussion on the web visit https://groups.google.com/a/mysociety.org/d/msgid/mysociety-community/ffa4aab6-ef4e-488b-b6ca-0cd1211c6d39%40mysociety.org.
For more options, visit https://groups.google.com/a/mysociety.org/d/optout.

[maxo...@gmail.com]

Wow, I got quite a kick out of seeing this!

I'm guessing the reason I appear as a 'hub' is that I've had the privilege of being able to travel to the UK and Taiwan multiple times and meet the civic hacking communities there, as well as getting to contribute to open source civic hacking projects for the last 5 years more or less full time starting at Code for America in 2010. Hopefully over the next 5 years a lot more people will get the opportunity to get paid to work on civic software and travel to meet other communities!

On Wednesday, November 18, 2015 at 1:36:56 PM UTC-8, Zarino Zappia wrote:

Interesting read!

Some clear parallels with my own MSc thesis from 2011, looking at participation in open data development (much of which is what we’d now call civic tech). My data was more limited, but even then, as with your research, certain names would popped up as intermediaries linking disparate communities. mySociety’s own Matthew Somerville comes to mind, or Friedrich Lindenberg at the OKF.

The link if you’re interested: http://zarino.co.uk/post/thesis

As you note, relying on the social features of Github is a poor proxy for actual community ties. I wonder whether you might get a more realistic picture by looking at contributors to repos? So the edges in your network diagram would, essentially, be shared github repositories. It would be less a network of “who’s following who”, and more “who’s actively working with who”.

Also, top designer tip: the data you’re trying to visualise on the Google map is much better suited to a chloropleth map, where you can shade countries/states based on volume of people, rather than using map markers which are prone to overlapping and which make it hard to visually compare areas.

Zarino Zappia

Designer, mySociety

@zarino

On 18 Nov 2015, at 17:40, Stefan Baack <s.b...@gmail.com> wrote:

Hi everyone,

I’m a PhD student from the Netherlands and I’m currently working on a research project about civic hacking for which I interviewed some members of mySociety — thanks again to everybody who took the time to talk with me!

In these interviews, I asked them to draw a map of all the different organizations or groups that are relevant for mySociety. These maps are very pretty interesting, and they inspired me to do a little experiment: compiling a list of civic tech organizations and scrape some information about them on GitHub. I drafted a blog post about my findings and I would love to hear your thoughts about it! You can find the drafted article here and the data from GitHub here if you’d like to take a closer look.

Best,
Stefan

​

--
You received this message because you are subscribed to the Google Groups "mysociety-community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mysociety-community+unsub...@mysociety.org.

[Stefan Baack]

Creating a contributor network is great idea. I just finished making one, it has some similarities (like the close ties between European and Latin American groups) but also interesting differences. I will refine the graph a bit and post it tomorrow!

On Wednesday, November 18, 2015 at 10:36:56 PM UTC+1, Zarino Zappia wrote:

Interesting read!

Some clear parallels with my own MSc thesis from 2011, looking at participation in open data development (much of which is what we’d now call civic tech). My data was more limited, but even then, as with your research, certain names would popped up as intermediaries linking disparate communities. mySociety’s own Matthew Somerville comes to mind, or Friedrich Lindenberg at the OKF.

The link if you’re interested: http://zarino.co.uk/post/thesis

As you note, relying on the social features of Github is a poor proxy for actual community ties. I wonder whether you might get a more realistic picture by looking at contributors to repos? So the edges in your network diagram would, essentially, be shared github repositories. It would be less a network of “who’s following who”, and more “who’s actively working with who”.

Also, top designer tip: the data you’re trying to visualise on the Google map is much better suited to a chloropleth map, where you can shade countries/states based on volume of people, rather than using map markers which are prone to overlapping and which make it hard to visually compare areas.

Zarino Zappia

Designer, mySociety

@zarino

On 18 Nov 2015, at 17:40, Stefan Baack <s.b...@gmail.com> wrote:

Hi everyone,

I’m a PhD student from the Netherlands and I’m currently working on a research project about civic hacking for which I interviewed some members of mySociety — thanks again to everybody who took the time to talk with me!

In these interviews, I asked them to draw a map of all the different organizations or groups that are relevant for mySociety. These maps are very pretty interesting, and they inspired me to do a little experiment: compiling a list of civic tech organizations and scrape some information about them on GitHub. I drafted a blog post about my findings and I would love to hear your thoughts about it! You can find the drafted article here and the data from GitHub here if you’d like to take a closer look.

Best,
Stefan

​

--
You received this message because you are subscribed to the Google Groups "mysociety-community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mysociety-community+unsub...@mysociety.org.

[Stefan Baack]

Thanks to everybody for the feedback and support. I just posted an updated version on my blog:

http://sbaack.com/2015/11/19/scraping-the-global-civic-tech-community-on-github-part-2.html

We can continue the discussion in the Poplus mailing list: https://groups.google.com/d/msg/poplus/kifWll-Dvm8/wpn0vuCUEgAJ

@Zarino This time I also added a contributor network and compare it with the follower network. Thanks for the suggestion and for the link to your thesis, will take a look! I hope you will forgive me for the lousy map again ;-)