developing the BL community

[Tom Cramer]

Fellow Blacklight adopters and fans, 

It's been exciting to see the growth of Blacklight--both the code base and the number of participating developers and institutions--over the last few quarters. Among the reasons Stanford decided to adopt Blacklight (in addition to its excellent functionality, proven flexibility, and solid architecture) was the Blacklight development group's eagerness and passion to run the software as an inclusive open source project. This gave us the confidence that we could contribute to shaping our own destiny, and be able to advance and sustain our chosen platform  with more than just Stanford's resources over the longterm. 

Stanford is now heavily invested in it, and committed to leveraging it as our access application for all forms of digital library resources. SearchWorks is now in production on Blacklight, and I think its quality and functionality demonstrate the merit of the decision to adopt Blacklight in the first place. 

And yet for all the features already in Blacklight to date, we recently did a roadmap of the things we'd like to add to SearchWorks, and came up with a two-page long list (single spaced). Working with just Stanford's resources, it would take us a couple of years to realize all the features on the list. But we're not working alone, and as the project grows, we'll certainly take advantage of development of items on our list that take place at other institutions. 

Given all this, I think it would be timely 

  1. to codify some of the principles that have guided Blacklight's development so far, and also 

  2. to make an effort to produce a coordinated roadmap so that the contributing individuals and institutions could make the most of each other's efforts, and 

  3. to use these to help enlist developer allies at new institutions

To kickstart this, here is a straw statement of principles for the Blacklight community and technical development. These are largely descriptive of how I believe things are working now (or in some cases how we'd like them to work), and I hope is neither too detail-light nor too heavyweight. That said, there is little in here that I (personally) am religious about--except the commitment to being inclusive, and the testing prerequisites--so don't hesitate to opine a different perspective. 

Mostly what I'd like to see is something we can use as a solid foundation on which to build a robust and long-term open source project, supported by a broad community of contributors. Would a statement such as this work for these purposes, and if not, how would you improve it? 

DRAFT Blacklight Technical & Community Principles

Overview

- Blacklight is an open source application for libraries (and anyone else), built on top of SOLR, and meant to deliver excellent access to all classes of information resources.

- Blacklight can ultimately be successful and sustainable in the long run only if it is an open project; that is, it takes contributions from a community of developers across many institutions to enhance and support it

- We will work to balance progress on Blacklight's codebase with open community discussion and transparent decision making as coequal goals

- Blacklight code is available through the Apache 2.0 open source license.

Technical Leadership

- Technical leadership of the project will be through a small group of proven developers who have demonstrated commitment to Blacklight's progress and success (and have commit rights to the source code)

- Committers must be: 

a. technically adept

b. constructive, positive members of the Blacklight software community

c. committed to producing useful, practical code for the community 

- To become a committer, candidates must be...

a. nominated by a current committer

b. voted on and approved by a majority of the current committers

c. committers may be voted out at any time by a (super?) majority of the other committers

- Committers will have a regular meeting, usually in the form of a conference call, to coordinate development & direction. 

- Releases will be vetted and controlled by a designated lead or leads. These roles may shift from release to release. 

Code Contributions & Principles

- the users of, interest in, resources, and talent pool of the Blacklight community will spread far beyond the developers on the committers list, and their institutions  

- Blacklight encourages and will facilitate taking code from contributors from many sources

- the structure of the source code management (soon GIT) will facilitate incorporating and using code from many sources

- Blacklight committers will actively take code contributions from non-committers and incorporate it into the code trunk 

 

- Working code wins

- You get what you give

- All contributed code must have full test coverage before it's committed. The current testing infrastructure is RSpec for everything but Rails views, and Cucumber for for Rails views.

- Tests must be committed at the same time code is. 

- All bugs and development tasks will be tracked in JIRA

- All code must be documented before it's committed

- (yada yada yada)

Roadmap & Transparency

- We will publish a roadmap to guide overall development. The items on this roadmap will be determined after an inclusive process of canvassing the wider Blacklight community, including code committers, contributors, users and a potential advisory board. 

- We envision regular/quarterly/semiannual/annual Blacklight convention of community members to help guide and galvanize new developments. These are likely to be appended to other community events, e.g., code4lib, for the sake of logistics.

- Tom

  | Tom Cramer

  | Associate Director, Digital Library Systems & Services

  | Stanford University LIbraries & Academic Information Resources

  | Stanford University

  | tcr...@stanford.edu

[Richard Green]

Tom

 

As a relative newbie to Blacklight I am not really in the best position to offer much constructive criticism on these suggestions; from my perspective they seem to be a good starting point for discussion.  However, I do take gentle exception to one part sentence:

 

“Blacklight is an open source application for libraries (and anyone else)…”

 

It’s the ‘libraries’ word that I’m uncomfortable with.  Whilst the statement is undoubtedly true I suggest that we should try to start this document with a phrasing that is more inclusive.  Already Blacklight is being considered and/or adopted by information providers who would not necessarily align themselves strongly with a library but who rather would see their contribution (for instance, an institutional repository) as part of their institution’s overall information provision alongside what is offered by their library.  (Let us acknowledge in passing the ability of Blacklight to surface a merged library/repository Solr index.)

 

I’m not entirely happy with this suggestion but I haven’t yet had enough caffeine this morning, something along the lines of “Blacklight is an open source application for information providers…”?  If you really want the word ‘library’ in there, then maybe “Blacklight is an open source application for information providers (libraries, repositories and others)…”?

 

Richard

___________________________________________________________________

 

Richard Green

Consultant to the University of Hull IT Systems Group

managing the CLIF and Hydra (Hull) Projects

 

http://edocs.hull.ac.uk

http://www.hull.ac.uk/clif

https://fedora-commons.org/confluence/display/hydra

[earlj moniz]

Hello Richard, Tom, and our favorite Latin lurker, et al.,
 . . . from the Blacklight documentation:

Blacklight is open source discovery software. Libraries (or anyone else) can use Blacklight to enable searching and browsing of their collections online.

I just love the term discovery software. . .
In this usage, it appears libraries means a book repository in the traditional sense.
In the usage presented by Tom, it appears he might mean libraries in a more general sense as a collection of things, objects, and most likely, digital collections of things.

Unlike Richard, I am a brand new newbie - not yet even qualified to call myself a newbie...

I do like the program; I love its intent; I love the fact that it is customizable; I love the community of users and contributors; and I especially like the friendliness and sense of sharing, teaching, and helping demonstrated by Matthew and Bess... so, I'll be around a long time - perhaps even long enough to become a full-fledged neophyte ... yeah, right, like that'll ever happen ... I may never be able to fully understand Linux, let alone the complexities of designing/modifying Ruby and Blacklight (grin) - I certainly know where to come for help when needed with Blacklight ... so there is hope for me in the long run...  (wink)
* * *
A good idea doesn't care who has it ... keep contributing, my new friend... every facet anyone puts on this Blacklight diamond makes it sparkle just that much brighter...
* * *
Until that time...  Earl J.

[Tom Cramer]

Richard, 

Your point is excellent--I don't believe that there is any need to constrain Blacklight's designated community to libraries, nor even libraries & repositories. To misquote CRIG, "the coolest thing to do with your application will be thought of by someone else."

Most simply, would something along these lines be workable and inclusive enough: 

Blacklight is an open source application, built on top of SOLR, and meant to deliver excellent access to all classes of information resources

?

- Tom

*****************************************************************************************
To view the terms under which this email is distributed, please go to http://www.hull.ac.uk/legal/email_disclaimer.html
*****************************************************************************************

[Tom Cramer]

Earl, 

Hello Richard, Tom, and our favorite Latin lurker, et al.,
 . . . from the Blacklight documentation:

At some point, you'll need to let on about your favorite Latin lurker. For now though, I'm enjoying the mystery. :) 

Blacklight is open source discovery software. Libraries (or anyone else) can use Blacklight to enable searching and browsing of their collections online.

I just love the term discovery software. . .
In this usage, it appears libraries means a book repository in the traditional sense.
In the usage presented by Tom, it appears he might mean libraries in a more general sense as a collection of things, objects, and most likely, digital collections of things.

Discovery is indeed a great term--it wraps up searching, browsing, known item finding and serendipity all in one term, and doesn't smack of jargon. One thing that we (locally) have in mind for Blacklight is bending it to also become a delivery application--i.e., the primary means through which a photo collection (say) could be searched, but then also the application where users can zoom, pan, rotate, manipulate, collect & share images of interest to them. 

Google Book Search is a great example of this type of hybrid application, I think. It makes it easy & seamless to search across the entire corpus, search within a single book, or read an entire book (if it's public domain...), all in one application. 

Locally, we seem to struggle between the boundaries of where discovery stops and delivery starts, and we often use the term disclivery to describe systems that consciously provide a mix of both functions. Access is another broader term which seems to effectively encompass both finding and getting/using. 

All that said, I suspect that most people will be attracted to Blacklight as a discovery tool, and perhaps it makes sense to keep its "mission statement" more focused than broad.

- Tom

[earlj moniz]

Hello Tom,
our favorite Latin lurker refers to the research term et al. as an indicator for and others ... since it is a Latin language derivation, and since we do have people who will come to lurk and rarely participate (there always are on any email distribution list), I use my phrase to recognize them and make sure they feel welcomed into the discussion - in spite of the fact that they rarely participate.
* * *
Disclivery begins to sound like a place where a person can drop off their old discs and horses to get them reshod . . .  (grin) ... of course, I'm not sure how many in the current generation would even recognize the term livery or livery stables as a place for horses and such . . .  (sigh) . . . I do like the term though ... unless there is another more widely used term ... I'll have to think about it longer...
* * *
The reason why I love the discovery term so much is because it is so expansive in its results and descriptive of the actions taken to bring those results...
A researcher enters a search term; the results bring back not only the files and records with that exact search term, but also other categories of results from the other search terms in the items sought that are included in other files and records not directly searched for initially... in that way, researchers find other related items that may be of use that were not initially sought or considered as significant or significantly related... so, in the process of doing the research, the researcher discovers other items that may be useful...
It would be nice if everyone used the same terms (catalogers, seekers, archivists, librarians), or at least the same set of terms ... unfortunately, that's not the way the individual human brain works ... so, in my mind, as it currently works, helping our historians and seekers find related files and records that may not be considered initially is a benefit and not a drawback.
* * *
For the delivery part, I'm not sure how many people these days sit at an Internet connected computer and expect it not to deliver graphics, text, and video in their searches. The delivery part is certainly an integral aspect considered in the planning and design of the system, but I'm not sure how significant a factor it becomes in the searching by patrons. If I search for images of Normandy beach on D-day, I am thrilled to see an option for video, but it may or may not help the search I'm on at the moment.
* * *
Those two paragraphs above certainly appear to be opposing views over the same issue - I like the discovery part, but not the introduction of results that distract?! Perhaps I'm just acknowledging the importance of both and haven't really decided yet, or perhaps I'm just a rambling moron with a computer handy and an email address to this Blacklight forum. . . OR maybe both! (grin)
* * *
In the long run, I think our discussion about a single term in the entire documentation about the Blacklight package is a bit nit-picky ... I do think it does lay the groundwork for newcomers or individuals who come to poke around to see what all the buzz is about over Blacklight. I don't think they will read one sentence, see the term discovery, library, or even disclivery, make up their minds, and leave without looking into it any further.

[Bess Sadler]

Tom just pointed out to me that I never responded to this on the list. I'm sorry about the radio silence. I have a bad habit of falling back on lazy consensus where silence == agreement, and with something like this that won't cut it. 

In short, I agree with this document. I think it's important for this project to have a life and a plan apart from the goals of any individual person or institution, and I believe a shared statement like this will help that happen. I agree with Richard's comment that we want something broader than "Blacklight is an open source application for libraries", since our scope has certainly grown to encompass digital repositories and other diverse data sets (hello, National Radio Astronomy Observatory folks!) and we want to reflect our commitment to serving those use cases. 

I'm a little torn on the subject of voting someone off the committers' list. Mostly I don't want to think about this because contemplating such a scenario is unpleasant, but the logical part of my brain realizes that it is important to discuss such an eventuality now, before it's needed, instead of attempting to hash it out during some sort of crisis, so here goes: 

I believe a simple majority should be enough to get you removed from the committers' list. In order to work effectively as a team we need to be able to trust the other members of our team, and if a majority of the group can't do that for any member of that group, that's enough to act on. However, as much as possible I'd like to remove any stigma from such a process. It's easy to envision someone who generally produces good code, but seems to be dropping the ball in a specific area, or during a specific time. I want us to be able to gently remove that person from committership, with a clear statement of why, and then give them the opportunity to remedy the situation and get added again. If removal of a committer is considered a nuclear option, we'll never use it. If instead it's a tool for keeping code quality high, and not considered to be either a punishment or a permanent step, we're giving ourselves a powerful tool for shaping our team and helping our contributors improve. The same should be true of someone wishing to join the committers' list for the first time -- if the group feels someone isn't ready yet, we should give them honest feedback about why, give them an opportunity to address any concerns, and reconsider the matter after some appropriate length of time. Both addition of a new committer and removal of an existing committer should be done by secret ballot, per guidelines at http://producingoss.com/en/committers.html#choosing-committers

But assuming such a vote, who votes? There are currently people who are technically code committers on the project who can't reasonably be considered active participants in this project's technical leadership. These include people who were active contributors at one time, but whose work has taken them in different directions, and people whom we trust who sometimes can help us out, but who don't usually have time to devote to this project. I'd like to follow the advice in "Producing Open Source Software" (http://producingoss.com/en/consensus-democracy.html#electorate)

Let's formalize the current de facto technical leadership as the people who participate in our Friday skype calls: 

- Vernon Chapman (US National Agriculture Library)

- Naomi Dushay (Stanford)

- Sean Hannan (JHU)

- Jessie Keck (Stanford)

- Matt Mitchell (UVA)

- Molly Pickral (UVA)

- Jason Ronallo (NCSU)

- Bess Sadler (UVA)

- Jennifer Vines (Stanford) 

Everyone else who currently has commit status should either be considered a non-voting contributor (people who still contribute sometimes but don't have time to play an active role right now) or removed from the project (people who have moved onto other work). 

I just re-read big chunks of Producing Open Source Software, and it's good stuff. As much as possible I'd like to model our governance on the Consensus Based Democracy model described at http://producingoss.com/en/consensus-democracy.html. 

Thanks for spearheading this, Tom.

Bess

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Blacklight Development" group.
To post to this group, send email to blacklight-...@googlegroups.com
To unsubscribe from this group, send email to blacklight-develo...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/blacklight-development?hl=en
-~----------~----~----~----~------~----~------~--~---

[Jason Ronallo]

I just wanted to say that I am in agreement with this document.
Blacklight has made some great technical advances to make it easier to
have a common codebase and many adopters. I'm glad to see a clear,
positive statement like this to formalize commitments to and goals for
the community.

I will echo what others have said about the applicability of
Blacklight to a broader group than just libraries. Blacklight makes it
easy now to take any metadata set and rather quickly have a search and
discovery interface within any Rails app.

Bess, I also like the attitude you express on governance. I read the
Consensus-base Democracy chapter from "Producing Open Source
Software." From what you wrote, do you have specific additions/changes
you'd like to see made to the "Blacklight Technical & Community
Principles"?

Jason

[Sean Hannan]

+1

Thanks, Tom, for putting this together. I think it covers a lot of what has already been understood amongst current developers while keeping it open and transparent for new adopters.

-Sean

---
Sean Hannan
Web Developer
Sherdian Libraries
Johns Hopkins University

| tcr...@stanford.edu<mailto:tcr...@stanford.edu>