Fellow Blacklight adopters and fans,

It's been exciting to see the growth of Blacklight--both the code base  
and the number of participating developers and institutions--over the  
last few quarters. Among the reasons Stanford decided to adopt  
Blacklight (in addition to its excellent functionality, proven  
flexibility, and solid architecture) was the Blacklight development  
group's eagerness and passion to run the software as an inclusive open  
source project. This gave us the confidence that we could contribute  
to shaping our own destiny, and be able to advance and sustain our  
chosen platform  with more than just Stanford's resources over the  
longterm.

Stanford is now heavily invested in it, and committed to leveraging it  
as our access application for all forms of digital library resources.  
SearchWorks is now in production on Blacklight, and I think its  
quality and functionality demonstrate the merit of the decision to  
adopt Blacklight in the first place.

And yet for all the features already in Blacklight to date, we  
recently did a roadmap of the things we'd like to add to SearchWorks,  
and came up with a two-page long list (single spaced). Working with  
just Stanford's resources, it would take us a couple of years to  
realize all the features on the list. But we're not working alone, and  
as the project grows, we'll certainly take advantage of development of  
items on our list that take place at other institutions.

Given all this, I think it would be timely
   1. to codify some of the principles that have guided Blacklight's  
development so far, and also
   2. to make an effort to produce a coordinated roadmap so that the  
contributing individuals and institutions could make the most of each  
other's efforts, and
   3. to use these to help enlist developer allies at new institutions

To kickstart this, here is a straw statement of principles for the  
Blacklight community and technical development. These are largely  
descriptive of how I believe things are working now (or in some cases  
how we'd like them to work), and I hope is neither too detail-light  
nor too heavyweight. That said, there is little in here that I  
(personally) am religious about--except the commitment to being  
inclusive, and the testing prerequisites--so don't hesitate to opine a  
different perspective.

Mostly what I'd like to see is something we can use as a solid  
foundation on which to build a robust and long-term open source  
project, supported by a broad community of contributors. Would a  
statement such as this work for these purposes, and if not, how would  
you improve it?

DRAFT Blacklight Technical & Community Principles

Overview
- Blacklight is an open source application for libraries (and anyone  
else), built on top of SOLR, and meant to deliver excellent access to  
all classes of information resources.
- Blacklight can ultimately be successful and sustainable in the long  
run only if it is an open project; that is, it takes contributions  
from a community of developers across many institutions to enhance and  
support it
- We will work to balance progress on Blacklight's codebase with open  
community discussion and transparent decision making as coequal goals
- Blacklight code is available through the Apache 2.0 open source  
license.

Technical Leadership
- Technical leadership of the project will be through a small group of  
proven developers who have demonstrated commitment to Blacklight's  
progress and success (and have commit rights to the source code)
- Committers must be:
        a. technically adept
        b. constructive, positive members of the Blacklight software community
        c. committed to producing useful, practical code for the community
- To become a committer, candidates must be...
        a. nominated by a current committer
        b. voted on and approved by a majority of the current committers
        c. committers may be voted out at any time by a (super?) majority of  
the other committers
- Committers will have a regular meeting, usually in the form of a  
conference call, to coordinate development & direction.
- Releases will be vetted and controlled by a designated lead or  
leads. These roles may shift from release to release.

Code Contributions & Principles
- the users of, interest in, resources, and talent pool of the  
Blacklight community will spread far beyond the developers on the  
committers list, and their institutions
- Blacklight encourages and will facilitate taking code from  
contributors from many sources
- the structure of the source code management (soon GIT) will  
facilitate incorporating and using code from many sources
- Blacklight committers will actively take code contributions from non-
committers and incorporate it into the code trunk

- Working code wins
- You get what you give
- All contributed code must have full test coverage before it's  
committed. The current testing infrastructure is RSpec for everything  
but Rails views, and Cucumber for for Rails views.
- Tests must be committed at the same time code is.
- All bugs and development tasks will be tracked in JIRA
- All code must be documented before it's committed
- (yada yada yada)

Roadmap & Transparency
- We will publish a roadmap to guide overall development. The items on  
this roadmap will be determined after an inclusive process of  
canvassing the wider Blacklight community, including code committers,  
contributors, users and a potential advisory board.
- We envision regular/quarterly/semiannual/annual Blacklight  
convention of community members to help guide and galvanize new  
developments. These are likely to be appended to other community  
events, e.g., code4lib, for the sake of logistics.

- Tom

   | Tom Cramer
   | Associate Director, Digital Library Systems & Services
   | Stanford University LIbraries & Academic Information Resources
   | Stanford University
   | tcra...@stanford.edu

Tom

As a relative newbie to Blacklight I am not really in the best position
to offer much constructive criticism on these suggestions; from my
perspective they seem to be a good starting point for discussion.
However, I do take gentle exception to one part sentence:

"Blacklight is an open source application for libraries (and anyone
else)..."

It's the 'libraries' word that I'm uncomfortable with.  Whilst the
statement is undoubtedly true I suggest that we should try to start this
document with a phrasing that is more inclusive.  Already Blacklight is
being considered and/or adopted by information providers who would not
necessarily align themselves strongly with a library but who rather
would see their contribution (for instance, an institutional repository)
as part of their institution's overall information provision alongside
what is offered by their library.  (Let us acknowledge in passing the
ability of Blacklight to surface a merged library/repository Solr
index.)

I'm not entirely happy with this suggestion but I haven't yet had enough
caffeine this morning, something along the lines of "Blacklight is an
open source application for information providers..."?  If you really
want the word 'library' in there, then maybe "Blacklight is an open
source application for information providers (libraries, repositories
and others)..."?

Richard

___________________________________________________________________

Richard Green

Consultant to the University of Hull IT Systems Group

managing the CLIF and Hydra (Hull) Projects

http://edocs.hull.ac.uk

http://www.hull.ac.uk/clif

https://fedora-commons.org/confluence/display/hydra

From: blacklight-development@googlegroups.com
[mailto:blacklight-development@googlegroups.com] On Behalf Of Tom Cramer
Sent: 16 September 2009 01:33
To: blacklight-development@googlegroups.com
Cc: Tom Cramer
Subject: [Blacklight-development] developing the BL community

Fellow Blacklight adopters and fans,

It's been exciting to see the growth of Blacklight--both the code base
and the number of participating developers and institutions--over the
last few quarters. Among the reasons Stanford decided to adopt
Blacklight (in addition to its excellent functionality, proven
flexibility, and solid architecture) was the Blacklight development
group's eagerness and passion to run the software as an inclusive open
source project. This gave us the confidence that we could contribute to
shaping our own destiny, and be able to advance and sustain our chosen
platform  with more than just Stanford's resources over the longterm.

Stanford is now heavily invested in it, and committed to leveraging it
as our access application for all forms of digital library resources.
SearchWorks is now in production on Blacklight, and I think its quality
and functionality demonstrate the merit of the decision to adopt
Blacklight in the first place.

And yet for all the features already in Blacklight to date, we recently
did a roadmap of the things we'd like to add to SearchWorks, and came up
with a two-page long list (single spaced). Working with just Stanford's
resources, it would take us a couple of years to realize all the
features on the list. But we're not working alone, and as the project
grows, we'll certainly take advantage of development of items on our
list that take place at other institutions.

Given all this, I think it would be timely

  1. to codify some of the principles that have guided Blacklight's
development so far, and also

  2. to make an effort to produce a coordinated roadmap so that the
contributing individuals and institutions could make the most of each
other's efforts, and

  3. to use these to help enlist developer allies at new institutions

To kickstart this, here is a straw statement of principles for the
Blacklight community and technical development. These are largely
descriptive of how I believe things are working now (or in some cases
how we'd like them to work), and I hope is neither too detail-light nor
too heavyweight. That said, there is little in here that I (personally)
am religious about--except the commitment to being inclusive, and the
testing prerequisites--so don't hesitate to opine a different
perspective.

Mostly what I'd like to see is something we can use as a solid
foundation on which to build a robust and long-term open source project,
supported by a broad community of contributors. Would a statement such
as this work for these purposes, and if not, how would you improve it?

DRAFT Blacklight Technical & Community Principles

Overview

- Blacklight is an open source application for libraries (and anyone
else), built on top of SOLR, and meant to deliver excellent access to
all classes of information resources.

- Blacklight can ultimately be successful and sustainable in the long
run only if it is an open project; that is, it takes contributions from
a community of developers across many institutions to enhance and
support it

- We will work to balance progress on Blacklight's codebase with open
community discussion and transparent decision making as coequal goals

- Blacklight code is available through the Apache 2.0 open source
license.

Technical Leadership

- Technical leadership of the project will be through a small group of
proven developers who have demonstrated commitment to Blacklight's
progress and success (and have commit rights to the source code)

- Committers must be:

            a. technically adept

            b. constructive, positive members of the Blacklight software
community

            c. committed to producing useful, practical code for the
community

- To become a committer, candidates must be...

            a. nominated by a current committer

            b. voted on and approved by a majority of the current
committers

            c. committers may be voted out at any time by a (super?)
majority of the other committers

- Committers will have a regular meeting, usually in the form of a
conference call, to coordinate development & direction.

- Releases will be vetted and controlled by a designated lead or leads.
These roles may shift from release to release.

Code Contributions & Principles

- the users of, interest in, resources, and talent pool of the
Blacklight community will spread far beyond the developers on the
committers list, and their institutions  

- Blacklight encourages and will facilitate taking code from
contributors from many sources

- the structure of the source code management (soon GIT) will facilitate
incorporating and using code from many sources

- Blacklight committers will actively take code contributions from
non-committers and incorporate it into the code trunk

- Working code wins

- You get what you give

- All contributed code must have full test coverage before it's
committed. The current testing infrastructure is RSpec for everything
but Rails views, and Cucumber for for Rails views.

- Tests must be committed at the same time code is.

- All bugs and development tasks will be tracked in JIRA

- All code must be documented before it's committed

- (yada yada yada)

Roadmap & Transparency

- We will publish a roadmap to guide overall development. The items on
this roadmap will be determined after an inclusive process of canvassing
the wider Blacklight community, including code committers, contributors,
users and a potential advisory board.

- We envision regular/quarterly/semiannual/annual Blacklight convention
of community members to help guide and galvanize new developments. These
are likely to be appended to other community events, e.g., code4lib, for
the sake of logistics.

- Tom

  | Tom Cramer

  | Associate Director, Digital Library Systems & Services

  | Stanford University LIbraries & Academic Information Resources

  | Stanford University

  | tcra...@stanford.edu

*************************************************************************** **************
To view the terms under which this email is distributed, please go to http://www.hull.ac.uk/legal/email_disclaimer.html
*************************************************************************** **************

Hello Richard, Tom, and our favorite Latin lurker, et al.,
 . . . from the Blacklight documentation:

*Blacklight <http://blacklight.rubyforge.org/classes/Blacklight.html> is
open source discovery software. Libraries (or anyone else) can use
Blacklight to enable searching and browsing of their collections online.*
I just love the term *discovery software*. . .
In this usage, it appears* libraries* means a book repository in the
traditional sense.
In the usage presented by Tom, it appears he *might* mean *libraries *in a
more general sense as a collection of things, objects, and most
likely, *digital
collections of things*.

Unlike Richard, I am a *brand new newbie* - not yet even qualified to call
myself a newbie...

I do like the program; I love its intent; I love the fact that it is
customizable; I love the community of users and contributors; and I
especially like the friendliness and sense of sharing, teaching, and helping
demonstrated by Matthew and Bess... so, I'll be around a long time - perhaps
even long enough to become a *full-fledged neophyte* ... yeah, right, like *
that'll* ever happen ... I may never be able to fully understand Linux, let
alone the complexities of designing/modifying Ruby and Blacklight (grin) - I
certainly know where to come for help when needed with Blacklight ... so
there is hope for me in the long run...  (wink)
* * *
*A good idea doesn't care who has it* ... keep contributing, my new
friend... every facet anyone puts on this Blacklight diamond makes it
sparkle just that much brighter...
* * *
Until that time...  Earl J.

Richard,

Your point is excellent--I don't believe that there is any need to  
constrain Blacklight's designated community to libraries, nor even  
libraries & repositories. To misquote CRIG, "the coolest thing to do  
with your application will be thought of by someone else."

Most simply, would something along these lines be workable and  
inclusive enough:

- Tom

On Sep 16, 2009, at 1:36 AM, Richard Green wrote:

Earl,

Google Book Search is a great example of this type of hybrid  
application, I think. It makes it easy & seamless to search across the  
entire corpus, search within a single book, or read an entire book (if  
it's public domain...), all in one application.

Locally, we seem to struggle between the boundaries of where discovery  
stops and delivery starts, and we often use the term disclivery to  
describe systems that consciously provide a mix of both functions.  
Access is another broader term which seems to effectively encompass  
both finding and getting/using.

All that said, I suspect that most people will be attracted to  
Blacklight as a discovery tool, and perhaps it makes sense to keep its  
"mission statement" more focused than broad.

- Tom

Hello Tom,
*our favorite Latin lurker* refers to the research term et al. as an
indicator for *and others* ... since it is a Latin language derivation, and
since we do have people who will come to lurk and rarely participate (there
always are on any email distribution list), I use my phrase to recognize
them and make sure they feel welcomed into the discussion - in spite of the
fact that they rarely participate.
* * *
*Disclivery* begins to sound like a place where a person can drop off their
old discs and horses to get them reshod . . .  (grin) ... of course, I'm not
sure how many in the current generation would even recognize the term livery
or livery stables as a place for horses and such . . .  (sigh) . . . I do
like the term though ... *unless* there is another more widely used term ...
I'll have to think about it longer...
* * *
The reason why I love the *discovery* term so much is because it is so
expansive in its results and descriptive of the actions taken to bring those
results...
A researcher enters a search term; the results bring back not only the files
and records with that exact search term, but also other categories of
results from the other search terms in the items sought that are included in
other files and records not directly searched for initially... in that way,
researchers find other related items that may be of use that were not
initially sought or considered as significant or significantly related...
so, in the process of doing the research, the researcher *discovers* other
items that may be useful...
It would be nice if everyone used the same terms (catalogers, seekers,
archivists, librarians), or at least the same set of terms ...
unfortunately, that's not the way the individual human brain works ... so,
in my mind, as it currently works, helping our historians and seekers find
related files and records that may not be considered initially is a benefit
and not a drawback.
* * *
For the *delivery* part, I'm not sure how many people these days sit at an
Internet connected computer and expect it not to deliver graphics, text, and
video in their searches. The delivery part is certainly an integral aspect
considered in the planning and design of the system, but I'm not sure how
significant a factor it becomes in the searching by patrons. If I search for
images of Normandy beach on D-day, I am thrilled to see an option for video,
but it may or may not help the search I'm on at the moment.
* * *
Those two paragraphs above certainly appear to be opposing views over the
same issue - I like the discovery part, but not the introduction of results
that distract?! Perhaps I'm just acknowledging the importance of both and
haven't really decided yet, or perhaps I'm just a rambling moron with a
computer handy and an email address to this Blacklight forum. . . OR maybe
both! (grin)
* * *
In the long run, I think our discussion about a single term in the entire
documentation about the Blacklight package is a bit nit-picky ... I
*do*think it does lay the groundwork for newcomers or individuals who
come to
poke around to see what all the buzz is about over Blacklight. I don't think
they will read one sentence, see the term *discovery*, *library*, or even *
disclivery,* make up their minds, and leave without looking into it any
further.

Until that time...  Earl J.

Tom just pointed out to me that I never responded to this on the list.  
I'm sorry about the radio silence. I have a bad habit of falling back  
on lazy consensus where silence == agreement, and with something like  
this that won't cut it.

In short, I agree with this document. I think it's important for this  
project to have a life and a plan apart from the goals of any  
individual person or institution, and I believe a shared statement  
like this will help that happen. I agree with Richard's comment that  
we want something broader than "Blacklight is an open source  
application for libraries", since our scope has certainly grown to  
encompass digital repositories and other diverse data sets (hello,  
National Radio Astronomy Observatory folks!) and we want to reflect  
our commitment to serving those use cases.

I'm a little torn on the subject of voting someone off the committers'  
list. Mostly I don't want to think about this because contemplating  
such a scenario is unpleasant, but the logical part of my brain  
realizes that it is important to discuss such an eventuality now,  
before it's needed, instead of attempting to hash it out during some  
sort of crisis, so here goes:

I believe a simple majority should be enough to get you removed from  
the committers' list. In order to work effectively as a team we need  
to be able to trust the other members of our team, and if a majority  
of the group can't do that for any member of that group, that's enough  
to act on. However, as much as possible I'd like to remove any stigma  
from such a process. It's easy to envision someone who generally  
produces good code, but seems to be dropping the ball in a specific  
area, or during a specific time. I want us to be able to gently remove  
that person from committership, with a clear statement of why, and  
then give them the opportunity to remedy the situation and get added  
again. If removal of a committer is considered a nuclear option, we'll  
never use it. If instead it's a tool for keeping code quality high,  
and not considered to be either a punishment or a permanent step,  
we're giving ourselves a powerful tool for shaping our team and  
helping our contributors improve. The same should be true of someone  
wishing to join the committers' list for the first time -- if the  
group feels someone isn't ready yet, we should give them honest  
feedback about why, give them an opportunity to address any concerns,  
and reconsider the matter after some appropriate length of time. Both  
addition of a new committer and removal of an existing committer  
should be done by secret ballot, per guidelines at http://producingoss.com/en/committers.html#choosing-committers

But assuming such a vote, who votes? There are currently people who  
are technically code committers on the project who can't reasonably be  
considered active participants in this project's technical leadership.  
These include people who were active contributors at one time, but  
whose work has taken them in different directions, and people whom we  
trust who sometimes can help us out, but who don't usually have time  
to devote to this project. I'd like to follow the advice in "Producing  
Open Source Software" (http://producingoss.com/en/consensus-democracy.html#electorate
)

Let's formalize the current de facto technical leadership as the  
people who participate in our Friday skype calls:

        - Vernon Chapman (US National Agriculture Library)
        - Naomi Dushay (Stanford)
        - Sean Hannan (JHU)
        - Jessie Keck (Stanford)
        - Matt Mitchell (UVA)
        - Molly Pickral (UVA)
        - Jason Ronallo (NCSU)
        - Bess Sadler (UVA)
        - Jennifer Vines (Stanford)

Everyone else who currently has commit status should either be  
considered a non-voting contributor (people who still contribute  
sometimes but don't have time to play an active role right now) or  
removed from the project (people who have moved onto other work).

I just re-read big chunks of Producing Open Source Software, and it's  
good stuff. As much as possible I'd like to model our governance on  
the Consensus Based Democracy model described at http://producingoss.com/en/consensus-democracy.html
.

Thanks for spearheading this, Tom.

Bess

On 15-Sep-09, at 8:32 PM, Tom Cramer wrote:

read more »

smime.p7s 6K Download

I will echo what others have said about the applicability of
Blacklight to a broader group than just libraries. Blacklight makes it
easy now to take any metadata set and rather quickly have a search and
discovery interface within any Rails app.

Bess, I also like the attitude you express on governance. I read the
Consensus-base Democracy chapter from "Producing Open Source
Software." From what you wrote, do you have specific additions/changes
you'd like to see made to the "Blacklight Technical & Community
Principles"?

Jason

Thanks, Tom, for putting this together. I think it covers a lot of what has already been understood amongst current developers while keeping it open and transparent for new adopters.

-Sean

---
Sean Hannan
Web Developer
Sherdian Libraries
Johns Hopkins University

On Sep 15, 2009, at 8:32 PM, Tom Cramer wrote:

Fellow Blacklight adopters and fans,

It's been exciting to see the growth of Blacklight--both the code base and the number of participating developers and institutions--over the last few quarters. Among the reasons Stanford decided to adopt Blacklight (in addition to its excellent functionality, proven flexibility, and solid architecture) was the Blacklight development group's eagerness and passion to run the software as an inclusive open source project. This gave us the confidence that we could contribute to shaping our own destiny, and be able to advance and sustain our chosen platform  with more than just Stanford's resources over the longterm.

Stanford is now heavily invested in it, and committed to leveraging it as our access application for all forms of digital library resources. SearchWorks is now in production on Blacklight, and I think its quality and functionality demonstrate the merit of the decision to adopt Blacklight in the first place.

And yet for all the features already in Blacklight to date, we recently did a roadmap of the things we'd like to add to SearchWorks, and came up with a two-page long list (single spaced). Working with just Stanford's resources, it would take us a couple of years to realize all the features on the list. But we're not working alone, and as the project grows, we'll certainly take advantage of development of items on our list that take place at other institutions.

Given all this, I think it would be timely
  1. to codify some of the principles that have guided Blacklight's development so far, and also
  2. to make an effort to produce a coordinated roadmap so that the contributing individuals and institutions could make the most of each other's efforts, and
  3. to use these to help enlist developer allies at new institutions

To kickstart this, here is a straw statement of principles for the Blacklight community and technical development. These are largely descriptive of how I believe things are working now (or in some cases how we'd like them to work), and I hope is neither too detail-light nor too heavyweight. That said, there is little in here that I (personally) am religious about--except the commitment to being inclusive, and the testing prerequisites--so don't hesitate to opine a different perspective.

Mostly what I'd like to see is something we can use as a solid foundation on which to build a robust and long-term open source project, supported by a broad community of contributors. Would a statement such as this work for these purposes, and if not, how would you improve it?

DRAFT Blacklight Technical & Community Principles

Overview
- Blacklight is an open source application for libraries (and anyone else), built on top of SOLR, and meant to deliver excellent access to all classes of information resources.
- Blacklight can ultimately be successful and sustainable in the long run only if it is an open project; that is, it takes contributions from a community of developers across many institutions to enhance and support it
- We will work to balance progress on Blacklight's codebase with open community discussion and transparent decision making as coequal goals
- Blacklight code is available through the Apache 2.0 open source license.

Technical Leadership
- Technical leadership of the project will be through a small group of proven developers who have demonstrated commitment to Blacklight's progress and success (and have commit rights to the source code)
- Committers must be:
a. technically adept
b. constructive, positive members of the Blacklight software community
c. committed to producing useful, practical code for the community
- To become a committer, candidates must be...
a. nominated by a current committer
b. voted on and approved by a majority of the current committers
c. committers may be voted out at any time by a (super?) majority of the other committers
- Committers will have a regular meeting, usually in the form of a conference call, to coordinate development & direction.
- Releases will be vetted and controlled by a designated lead or leads. These roles may shift from release to release.

Code Contributions & Principles
- the users of, interest in, resources, and talent pool of the Blacklight community will spread far beyond the developers on the committers list, and their institutions
- Blacklight encourages and will facilitate taking code from contributors from many sources
- the structure of the source code management (soon GIT) will facilitate incorporating and using code from many sources
- Blacklight committers will actively take code contributions from non-committers and incorporate it into the code trunk

- Working code wins
- You get what you give
- All contributed code must have full test coverage before it's committed. The current testing infrastructure is RSpec for everything but Rails views, and Cucumber for for Rails views.
- Tests must be committed at the same time code is.
- All bugs and development tasks will be tracked in JIRA
- All code must be documented before it's committed
- (yada yada yada)

Roadmap & Transparency
- We will publish a roadmap to guide overall development. The items on this roadmap will be determined after an inclusive process of canvassing the wider Blacklight community, including code committers, contributors, users and a potential advisory board.
- We envision regular/quarterly/semiannual/annual Blacklight convention of community members to help guide and galvanize new developments. These are likely to be appended to other community events, e.g., code4lib, for the sake of logistics.

- Tom

  | Tom Cramer
  | Associate Director, Digital Library Systems & Services
  | Stanford University LIbraries & Academic Information Resources
  | Stanford University
  | tcra...@stanford.edu<mailto:tcra...@stanford.edu>