Time to vote on features

[Martin Haye]

Dear XTF users,

We've collected quite a few suggestions now on the development ideas page, so I think it's time to do a first round of voting to see which ones Kirk and I should work on first. Each of you on this mailing list gets a vote, except employees of CDL.

I've come up with a simple voting system that I hope will be equitable.

Review the list of items below. Pick your "A" list, meaning things you definitely want right now, and also pick a "B" list meaning things you'd probably use in the future. Then reply to the list with your vote, or if you are shy you can send directly to me.

Here is a sample vote (totally fictitious):

A - important now: #4, #8, #12
B - would be nice in the future: #3, #15, #7

Get your vote in this week (before midnight Sunday Oct 21). I will tabulate the votes, break ties myself, and announce the results next week. If you miss this round of voting, we'll do another in a few months.

Here are the 15 features to consider, each with its own number. That way you can use the numbers when voting:

1. Revise the default stylesheets to demonstrate more XTF features, particularly: faceted browsing; AJAX; query routing and separate indexes; other? 

2. Add some sort of "advanced" query mode, where users could type in a boolean query instead of filling out fields on a form. This could be based on CQL or on Lucene's built-in query language. CQL would be a bit easier as we already have most of the Java mechanics created already.

3. Alternatively,allow a direct hook to step 3 of the query process, so that forms and query parser can be in another language, ie php etc. 

4. Add an OBDC or other Db connection for meta data, to index meta data directly from a mysql database etc.,ie, "SELECT * IN  meta WHERE  docid='$docid'" 

5. Add a mechanism to extend plural folding, by:

• Folding many words - in order to permit an entire verb paradigm etc.
• allow end users to decide whether to use the feature on any particular search. (these changes would be excellent for foreign language searches).

6. Start a wiki where XTF users can share their own tips and tricks. Perhaps this would be as simple as creating a new page for the Google Group? 

7. Add support for Snowball Stemmers for Lucene

8. Add Search-in-document box and Print View to EAD interface (similar to TEI interface) 

9. Add a config option to allow snippets to go past sectionType boundaries. Useful for applications in which sectionType is used at a low level on small pieces of text.

10. Add a total snippets counter to be able to place along side the total document counter at the head of the results page.  This type of data is quite useful for getting statistics on authors or works to compare them.

11. Add ability to mark books, e-mail, store, or export citations.

12. Improve browsing: be able to jump to a particular letter.

13. Be able to link from individual book back to the search results list from which it was chosen.

14. Be able to restore the red-highlighted "hits" after they have been cleared from the Table of Contents. 

15. Be able to use lucene fuzzy search (this would be useful to improve the search in brute ocr output)

[Martin Haye]

Jamie requests that we all consider two additional features. Since nobody has had a chance to vote yet, this seems fine to me. Here they are:

16. Be able to index a file as multiple documents

17. Interact with the underlying engine via HTTP and XML, a la Solr

--Martin

[Seth Cherney]

A: 4. 5. 10. 16.
B: 2. 3. 11. (partic "store")

Thanks, Seth


Martin Haye <m...@snyder-haye.com> wrote:

---

Yahoo! oneSearch: Finally, mobile search that gives answers, not web links.

[Jamie Orchard-Hays]

Seems like 6 doesn't really qualify as a feature request. :-)

Jamie

[Jamie Orchard-Hays]

Could you expound on this? I am not clear what the context is, nor what it means to mark them.

Jamie

On Oct 15, 2007, at 6:39 PM, Martin Haye wrote:

[Jamie Orchard-Hays]

3 and 17 are related. As XTF exists now, you could actually create your own form, say in Ruby on Rails, send a GET to XTF with the raw flag set to true and get the XML back to parse and display yourself. This is sort of the first step towards what 3 and 17 are asking.

I would love to be able to post a document to XTF to index it, which is one of the things I mean by 17.

Jamie

On Oct 15, 2007, at 6:39 PM, Martin Haye wrote:

3. Alternatively,allow a direct hook to step 3 of the query process, so that forms and query parser can be in another language, ie php etc.  

[Martin Haye]

Well, starting a wiki on some other site (e.g. pbwiki, or hacking one together for SourceForge) would be quite a bit of work for us, even if it didn't involve changes to XTF itself. So I think it is a valid thing to think about doing.

--Martin

[Jamie Orchard-Hays]

Definitely! I could take this on, if you don't need to keep it in house at CDL.

Jamie

[elel]

I only have an A list since I am just getting started.

8, 4, & 1.

Lynn

[Jamie Orchard-Hays]

Bump. I never got a reply to this and am still curious so I can go ahead and vote.

[Martin Haye]

Item 11 was contributed by Sheila Yeh... maybe she can reply in more detail. I'm kind of assuming she means a "book bag" where you can mark search results, then email the list, or store it for future reference, or export it in some format that citation software can read. Sheila?

--Martin

On 10/16/07, Jamie Orchard-Hays <ja...@dang.com> wrote:

[Jamie Orchard-Hays]

Ah, ok. That's something I have actually built in our Solr-powered biblio application. It'll email, or export in text and EndNote formats.

[Steve Toub]

Hi Jamie--

For that functionality, what language is that written in and is the code sharable?
    --SET

--
Steve Toub
California Digital Library
(510) 987-0078

[Jamie Orchard-Hays]

Hey Steve. It's for a proprietary app I'm building that hasn't been
released yet. It's Ruby and Ruby on Rails code. At some point the
code might be sharable, but not at the moment.

This makes me think again how desirable decoupling XTF's indexing/
searching into a pluggable architecture, like Solr.

[Sheila Yeh]

Hi Jamie, Martin,

I missed responding to your email earlier during the day.  I have asked Meg Spernoga, our Assistant Director of Library Systems and User Services to reply and am sure that she will do so tomorrow.  It is in line with what you’ve built and described.

 

Thanks,

Sheila

 

---

[Sheila Yeh]

Our votes, OhioLINK:

 

A choices:

11. Add ability to mark books, e-mail, store, or export citations.
12. Improve browsing: be able to jump to a particular letter.
13. Be able to link from individual book back to the search results list from which it was chosen.

1. Revise the default stylesheets

2. Add some sort of "advanced" query mode

 

B choice:

14. Be able to restore the red-highlighted "hits" after they have been cleared from the Table of Contents. 

 

Thanks,

Sheila

 

---

From: xtf-...@googlegroups.com [mailto:xtf-...@googlegroups.com] On Behalf Of Martin Haye
Sent: Monday, October 15, 2007 5:08 PM
To: xtf-...@googlegroups.com
Subject: [xtf-user] Re: Time to vote on features

 

Jamie requests that we all consider two additional features. Since nobody has had a chance to vote yet, this seems fine to me. Here they are:

[Seth Cherney]

not quite.  I mean to bypass the processing of the form before it is passed to lucene.  also, I dont want to use raw output, I am happy with the results formatting and the doc display, find the weak point of xslt is in the forms.

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

[Seth Cherney]

otherwise, someone could perhaps be given a limited environment on your site to admin a wiki?  even give 2 or 3 people this possibility?  that is the point of a wiki after all, distribute the work load.

really, since we arent doing anything fancy, pmwiki is about 2 hours to set up for me.

seth

[BT]

A: 3, 16
B: 2, 7, 15

[David Lowe]

UConn would propose the following:

#18 Baked in integration with a METS-aware page image turner, featuring
a display option toggle from page image display to text display (cp.
latest vsn. of DLXS/MBooks interface or Harvard's PDS)

That would be our A list.

Our B list includes the following user experience enhancements, in
priority order:
#8
#2
#13
#11
#12

Thanks for asking!
--DBL

[Jill Glaser]

A:  1, 12, 13, 18
B:  2, 8, 11

[Fabio Ciotti]

As a general note I think that the features the dev team should work
on first should be those which involve Java programming. Many ideas in
the list can or should be achieved only with XSLT programming, which
is a task that is left to adopters and that can be shared amongs
them... provided we have a wiky :-).

Anyway my 2 cents...

A: 9, 5, 15, 7
B: 17, for example developing a REST interface...

Fabio

[Jakob Saternus]

Better late than never..

A - 1, 4, 7
B - 6, 10, 12, 13

Regards,
Jakob

Tuesday, October 16, 2007, 12:39:59 AM, you wrote:

> Dear XTF users,

> We've collected quite a few suggestions now on the development ideas

> page<http://groups.google.com/group/xtf-user/edit/development-ideas?>,

> - Folding many words - in order to permit an entire verb paradigm etc.

> - allow end users to decide whether to use the feature on any

> particular search. (these changes would be excellent for foreign language
> searches).

> 6. Start a wiki where XTF users can share their own tips and tricks. Perhaps
> this would be as simple as creating a new page for the Google Group?

> 7. Add support for Snowball Stemmers for

> Lucene<http://lucene.apache.org/java/docs/lucene-sandbox/>

[Jamie Orchard-Hays]

A list:

16. Be able to index a file as multiple documents
17. Interact with the underlying engine via HTTP and XML, a la Solr 

3. Alternatively,allow a direct hook to step 3 of the query process, so that forms and query parser can be in another language, ie php etc.  

4. Add an OBDC or other Db connection for meta data, to index meta data directly from a mysql database etc.,ie, "SELECT * IN  meta WHERE  docid='$docid'"  

B list:

11. Add ability to mark books, e-mail, store, or export citations. 

[Seth Cherney]

I'm leaving for a week on an archeology trip to Sinai.  Thanks to all for the contribs on this one, looks like xtf will be taking something of a leap forward :-). 

I agree with Fabio that we should set up pages on the wiki to let us follow up on the items that don't get finished, although I have high hopes that there will be only a few (ie, thanks core XTF Devs!)

Perhaps next time we should classify small and large things as well as java and xslt, and then perhaps some users could volunteer to take on a few of the items, to permit the XTf core team to focus on the items we cant knock out quickly among us.

Seth

[Jamie Orchard-Hays]

I did get the PBWiki set up a few days ago:

<http://xtf-user.pbwiki.com/>

[Jiao, Dazhi]

My vote:

A -
#1 Revise the default stylesheets to demonstrate more XTF features,

particularly: faceted browsing; AJAX; query routing and separate
indexes; other?

#2 Add some sort of "advanced" query mode, where users could type in a

boolean query instead of filling out fields on a form. This could be
based on CQL or on Lucene's built-in query language. CQL would be a bit
easier as we already have most of the Java mechanics created already.

#3 Alternatively,allow a direct hook to step 3 of the query process, so

that forms and query parser can be in another language, ie php etc.

B-
#9 Fabio suggests: Add a config option to allow snippets to go past

sectionType boundaries. Useful for applications in which sectionType is
used at a low level on small pieces of text.

#12 Improve browsing: be able to jump to a particular letter.
#15 Be able to use lucene fuzzy search (this would be useful to improve

the search in brute ocr output)

Thanks!

> 3. Alternatively,allow a direct hook to step 3 of the query process, so that
> forms and query parser can be in another language, ie php etc.
>
> 4. Add an OBDC or other Db connection for meta data, to index meta data
> directly from a mysql database etc.,ie, "SELECT * IN meta WHERE
> docid='$docid'"
>

> 5. Add a mechanism to extend plural folding, by:
>
> - Folding many words - in order to permit an entire verb paradigm etc.
>
> - allow end users to decide whether to use the feature on any
> particular search. (these changes would be excellent for foreign language
> searches).
>
> 6. Start a wiki where XTF users can share their own tips and tricks. Perhaps
> this would be as simple as creating a new page for the Google Group?
>
> 7. Add support for Snowball Stemmers for
> Lucene<http://lucene.apache.org/java/docs/lucene-sandbox/>
>
> 8. Add Search-in-document box and Print View to EAD interface (similar to
> TEI interface)
>
> 9. Add a config option to allow snippets to go past sectionType boundaries.
> Useful for applications in which sectionType is used at a low level on small
> pieces of text.
>
> 10. Add a total snippets counter to be able to place along side the total
> document counter at the head of the results page. This type of data is
> quite useful for getting statistics on authors or works to compare them.
>

> 11. Add ability to mark books, e-mail, store, or export citations.
>

> 12. Improve browsing: be able to jump to a particular letter.
>

> 13. Be able to link from individual book back to the search results list
> from which it was chosen.
>
> 14. Be able to restore the red-highlighted "hits" after they have been
> cleared from the Table of Contents.
>
> 15. Be able to use lucene fuzzy search (this would be useful to improve the
> search in brute ocr output)
>
> >
>
>

--
Dazhi (David) Jiao
System Analyst and Programmer
Digital Library Program
Indiana University at Bloomington
1320 E 10th Street, 501,
Bloomington Indiana 47405
Tel: 812-856-0089 Email: dj...@indiana.edu