Building a website for crowdsourcing transparency tools

[Jennifer Bell]

The goal of visiblegovernment.ca is to become a catalyst for the
creation of online tools for government transparency. The site should
be able to capture ideas raised at networking events, or ones that
people come up with on their own, in their basement. It should allow
people to form groups of interest around the tools, donate to fund the
tools, and see them through to implementation.

In my research on comprable sites, I found a good review of
crowdsourcing sites here:
http://www.readwriteweb.com/archives/crowdsourced_workforce_guide.php

Of those, the design I think is the most applicable is here:
www.cofundos.org

But it needs a lot more of a front end for people to collaboratively
plan and agree what the tools would do, which is, I think, a very
large part of the problem.

Also, (inspired by a comment from Hugh McGuire) the site could
incorporate contests and bounties, where people can come together to
pool money for a cash prize for designs on a particular topic.

A good 'prize/contest' style site is here:
http://mydreamapp.com/

I've been working (slowly) on a ruby app to do the above. I
gravitated towards ruby on rails as opposed to php because of the
simplicity of the REST and database interfaces, and the integrated
testing tools.

Jennifer

[Jennifer Bell]

As mentioned, one part of visiblegovernment.ca will be a website that
will be a catalyst for getting online transparency tools up and
running as quickly and effectively as possible. The stages I had
planned for tool development on the site are below. I am very
interested in feedback on this process. Do you think it will work?
Is there a better way?

1. Proposal

- author describes idea
- people can show support for a proposal by joining/supporting it

2. Planning

- proposals are paired with experienced software developers who
volunteer to collaborate on use case development, which should include
a thorough description of what the tools is to do, and the acceptance
test cases. Planners will be requested to keep iterations to
approximately two months, to keep things moving.

Note: I was thinking that in the first iteration of the
visiblegovernment.ca site, only a limited number of ideas -- say, 7 or
so -- would be promoted from proposal to planning, to keep down the
amount of noise on the site. Selection would be made based on
indicated level of support.

3. Bidding

- Once a site has been planned, it will be passed to web development
firms to estimate. Firms will also be allowed to register to be
notified when donations reach a certain amount. Project supporters
can make donations.

4. Implementation

- Web firm develops site

5. Testing

- Project members who are designated as testers can vote on which
acceptance test cases pass. Donated prize will be awarded based on
percentage of test cases passing.


Jennifer

[Cory Horner]

Sounds reasonable, but after the consultant builds the tool, what
happens?

Who operates the site, and adds features? Is the consultant still
around, or are volunteers doing their best to figure out how what the
consultant built works and making it do what they originally dreamed up?

Cheers,
Cory.

On 7-Jul-08, at 11:32 AM, Jennifer Bell wrote:

> ...The stages I had

> planned for tool development on the site are below. I am very
> interested in feedback on this process.

> 1. Proposal
>
> - author describes idea
> - people can show support for a proposal by joining/supporting it
>
> 2. Planning
>
> - proposals are paired with experienced software developers who
> volunteer to collaborate on use case development, which should include
> a thorough description of what the tools is to do, and the acceptance
> test cases. Planners will be requested to keep iterations to
> approximately two months, to keep things moving.
>

> ...

[Jennifer Bell]

Good points.

1. Who operates the site?

- I thought the sites could be hosted and managed on a common
account.

2. Who adds features?

- After the first iteration is accepted, the planning process opens up
again for starting the next iteration. The planners agree on what
features are to go in to the next iteration. Web development firms
would then bid on implementation. If a volunteer open source
developers want to get involved at this point they could, perhaps
bidding on the project as 'free' and giving a time estimate. The key
for keeping the product stable is that each iteration defines test
cases, and all test cases pass on each iteration before money is
awarded.

All this hinges on the idea that 1) enough people contributing to the
planning stage are savy enough to be able to properly describe their
iterations and test cases in advance, and 2) that the process will
lead to a better planned / more stable result than rushing out and
doing it.

While the second point is perhaps debateable, I do think that money
has to be a part of the process, somehow, in order to get reasonably
rapid and directed progress on a tool. I was impressed recently by
the fact that even the Mozilla foundation gets 600K/year from Google
to house a stable of core developers.

Another option is to scrap the above and just design the site entirely
around collaborative contests and prizes.... Would be interested in
ideas on how that might work, and if it would be better or worse.

Jennifer

[ConstructivEthos]

We are privileged, we build on the backs of giants. Much trial and
error on using the internet to open democracies has been done in the
USA and UK. We can use their knowledge to create a process and
standard to effectively open the Canadian government. These
discussions should be the basis of that process and those standards.
The end result should be a clear and thorough mission statement and
web site design standard outlining our aims and how they can be
achieved.

I'm sure everyone is aware of some of these points and may even agree
with them but I think it is important to spell out the process as
clearly & thoroughly as possible to have the best chance for success.
I have summarized the ideas at the end of this post.

It is very important to have someone being paid to direct a project.
Completing projects that match a vision is very difficult especially
the polishing portion and that polish greatly affects how and how many
people use the site. The primary person can be supported by
volunteers but the primary programmer should be paid to focus on the
completion and execution.

I imagine one of the project constraints is development costs
especially on the initial projects. A good option is to hire students
or new graduates to work on the development. The work is slower but
you get a skilled, passionate and committed worker.

The hard part of creating websites is determining how to program your
vision into reality. Well noted and explained code can be understood
after it is written. Part of the technical document describing the
project should require good notes both in code and out. These notes
will make future changes and upgrades easier.

If the documentation is good and the changes are not too complicated
then the person volunteering to run the site could make upgrades. On
more complicated sites or when the number of sites is large or when
the upgrade is substantial it may be necessary to hire some one.

Providing a clear idea as to the design & function of the site will
help the programmers minimize development time. The design focus
should be defining how the site will work & how users will interact
with the site not how it will be programmed. To that end a good
amount of time should be spent determining how & why the projects will
work. I suggest we enlist the help of people who specialize in web
site design as well as those who specialize in coding. As design is
easier to review then code we should be able to do this with
volunteers.

A key part of opening government is making information available in
accessible & structured formats. All of the sites should present
their valuable information in accessible & structured formats and
should have readily available links to that information. This should
be another basic requirement within the technical documents describing
each project.

Summary
1. Create a mission statement and web site design standard
2. Pay the primary programmer (at least)
3. Consider a student / new graduate as the primary programmer
4. Require clear documentation (in code and out) on how the code
functions
5. Easier upgrades are completed by volunteers
6. Complex upgrades are hired out if necessary
7. Clearly define design & function of the site
8. Enlist web site design specialists (volunteers ideally)
9. Valuable information should be accessible & structured

In other news, I'm listening to DJ Champion out of Montreal, you
should check them out, and you should YouTube "Alice Pogo".

Cheers,
CE

[ConstructivEthos]

At the moment, it seems there are almost no good tools and not easily
accessible & structured information to allow the opening of the
Canadian government. That information will be the foundation of
everything else that is built and needs to be rock solid. Stability
comes through planning and modification.

CE

[Jennifer Bell]

> Summary
> 1. Create a mission statement and web site design standard

Yes, the mission statement is key. For maximum flexability, sites
should be small in scope and easily work together.

> 2. Pay the primary programmer (at least)
> 3. Consider a student / new graduate as the primary programmer

A primary programmer, at least, is key. Ultimately, someone has to
have a trackable reputation in the system if money is involved.

> 4. Require clear documentation (in code and out) on how the code
> functions

For enterprise class, multi-layer software, it's true that a lot of
external code documentation is required. For newer frameworks like
ruby or django, maybe not so much. I would prefer to see emphasis on
test cases, which must not break as the code base moves forward, and
provide implicit examples of how to use the code (see Test Driven
Development), and internal API documentation.

What are some documentation standards in the open source world? An
common standard could be written into the development contract
acceptance criteria.

> 5. Easier upgrades are completed by volunteers

Hm. I think this is a key difference between a programmer-driven
project and a user-driven project. If it's a community-driven (and
funded) effort, the community ought to agree on what upgrades are
important or neccessary, to prevent the code base they paid for going
off on a tangent b/c of a single person's viewpoint.

As I said above, users/planners should agree on the next iteration
before coding starts, even if volunteers are involved. Ad-hoc
upgrades ought to be discouraged.

> 6. Complex upgrades are hired out if necessary
> 7. Clearly define design & function of the site

Yes. There should be a common, repeatable framework for defining what
the site does. eg.

- Mission
- Questions the site answers
- Data the site collects and shares

> 8. Enlist web site design specialists (volunteers ideally)

Yes. I thought that, after the concept stage, projects that receive a
lot of interest are paired with a distributed volunteer technical team
for planning. (Check out the contest site http://mydreamapp.com/,
which follows a similar model -- except the planning team is paid.)
The planning team uses a wiki-like tool that ultimately generates a
fixed 'iteration' document, which is then bid on by competing firms to
ensure lowest cost implementation, and effective use of donor dollars.

What would be truly amazing would be to have free access to a
dedicated online website planning tool like www.protoshare.com. I
asked about a free account (it would be a good promotional tool for
them), but they haven't written back.

> 9. Valuable information should be accessible & structured

Absolutely, that should be a stipulation of the funding contract.

Sorry for the delayed response, I took some time off to think and then
didn't get back to it. Please keep the ideas coming.

Jennifer

[Jennifer Bell]

There's a toe-hold. For instance, the hospitality and expense data
can be scraped from web pages. Also (though this isn't a current
project for VG) most of the GoC site could be probably be scraped,
uploaded to a different site and made searchable, with RSS feeds for
changes. I think this is the basis of govtrack.org, which as I
understand, was initially coded by one person -- a grad student -- in
his spare time. The scrapers, etc. for that site were open-sourced a
few weeks ago.

Tools that provide benefits to the public using what's available ought
to be enough to start pushing the concepts forward. The key is
choosing the projects to start off with.

So you know, the VG pilots were chosen for maximum appeal to
journalists. The idea is that journalists will drive publicity, thus
driving users.

Jennifer

[Jennifer Bell]

I finished reading Clay Shirkey's book, 'Here Comes Everybody' last
week. It had a nice section on the development of a wikipedia page as
being a slow accretion of good ideas. He describes a wikipedia page
as having an implicit promise: "If you help, this will get better".

In that spirit, I've posted the first draft of a description of a
crowd-sourcing website in the 'file' section of this group. I think
that opening up the process of proposing, funding, and building, is a
good idea in the longer term -- it will increase access to funding,
allow more tools to be proposed and developed, and increase the
transparency of the process. However, there's also the risk that it
won't work.

Right now, in the talks where I'm trying to get funding, I'm currently
pushing the idea of a crowd-sourced website into the background behind
other ideas of community building, pilot projects, and traditional
grant distribution. I think the key in the short term is to get some
tools out of the gate to get the process rolling.

Jennifer

On Jul 16, 10:51 am, Jennifer Bell <visiblegovernm...@gmail.com>
wrote: