search in the tiddlyweb ecosystem

[chris...@gmail.com]

I've written up some notes on search in the tiddlyweb ecosystem and
some areas in which reality may fall short of expectations or hopes.

http://cdent.tiddlyspace.com/SearchShortcomings

That writing needs some feedback so there can be further developments
based on the feedback (if any).

Thanks.

--
Chris Dent http://burningchrome.com/
[...]

[cdent]

On Monday, May 7, 2012 5:06:24 PM UTC+1, cdent wrote:

I've written up some notes on search in the tiddlyweb ecosystem and
some areas in which reality may fall short of expectations or hopes.

   http://cdent.tiddlyspace.com/SearchShortcomings

I've got a working version of /hsearch installed on tiddlyspace.com, described at http://cdent.tiddlyspace.com/20120510 

[cdent]

On Thursday, May 10, 2012 1:47:40 PM UTC+1, cdent wrote:

I've got a working version of /hsearch installed on tiddlyspace.com, described at http://cdent.tiddlyspace.com/20120510 

This seems to be working pretty well. Has anyone else tried it?

I used it recently to remind myself where information on deleting a space was located. Compare these results:

http://tiddlyspace.com/search?q=delete

http://tiddlyspace.com/hsearch?q=delete 

In the former the desired results doesn't show up at all, in the latter it is the 11th result.

[Ben Gillies]

On Tue, May 15, 2012 at 5:40 PM, cdent <chris...@gmail.com> wrote:
>
> On Thursday, May 10, 2012 1:47:40 PM UTC+1, cdent wrote:
>>
>> I've got a working version of /hsearch installed on tiddlyspace.com,
>> described at http://cdent.tiddlyspace.com/20120510
>
>
> This seems to be working pretty well. Has anyone else tried it?

It doesn't seem to respect ControlView. For example, if I search my
own space for:

http://bengillies.tiddlyspace.com/hsearch?q=css

I get a lot of tiddlers from other spaces.

[chris...@gmail.com]

On Tue, 15 May 2012, Ben Gillies wrote:

> It doesn't seem to respect ControlView. For example, if I search my
> own space for:
>
> http://bengillies.tiddlyspace.com/hsearch?q=css
>
> I get a lot of tiddlers from other spaces.

Yes, thank goodness, ControlView[1], except in rare cases, is pretty
much a censor, limiting exploration, learning, discovery, etc.

As an "h for human" search, hsearch is for people to find things on
tiddlyspace (all of it), so the control view limitations seem out of
place.

It does, however, list the tiddlers in their proper spaces (by URI), so
the protect the pope issue that ControlView was created to protect
against isn't a problem[2].

If you want to limit the results to a bag you can:

http://bengillies.tiddlyspace.com/hsearch?q=bag:bengillies_public%20css

(Most of the same <something>: limiters used in the other search will
work with hsearch.)

[1] Speaking here of just ControlView, not DropPrivs, which has a
different purpose related to safety.

[2] In fact because of the way the a tiddlers URI is now calculated on
TiddlySpace, using controlview on /search could be turned off, as the
links will go to the tiddlers in their "home" space. It used to be
that they would not, thus /search was covered by ControlView. Now,
however, it ought to be possible to have ControlView cover just
/bags* and /recipes* based URIs (friendly uris are already "correct"
by their nature).

ControlView kicks in based on a test of "is it this, this or this".
/search is currently in the truthy side of that, /hsearch is not,
simply because I didn't add it in.

[Ben Gillies]

On Tue, May 15, 2012 at 6:16 PM, <chris...@gmail.com> wrote:
> On Tue, 15 May 2012, Ben Gillies wrote:
>
>> It doesn't seem to respect ControlView. For example, if I search my
>> own space for:
>>
>> http://bengillies.tiddlyspace.com/hsearch?q=css
>>
>> I get a lot of tiddlers from other spaces.
>
>
> Yes, thank goodness, ControlView[1], except in rare cases, is pretty
> much a censor, limiting exploration, learning, discovery, etc.

I get what you're saying, but I think there's good reason for it being
applied as normal. That is:

1) It's applied to everything else
2) If I search the bengillies space I expect results from the
bengillies space only, unless I explicitly opt out (as per the
X-ControlView header)
3) If I want to search across the whole of TiddlySpace, I can do so at
tiddlyspace.com

> As an "h for human" search, hsearch is for people to find things on
> tiddlyspace (all of it), so the control view limitations seem out of
> place.

I'd say it's for people to find things without having to remember the
syntax. If they're searching in a space, I'd expect results to be
limited to just that space. Likewise if they search tiddlyspace.com
(i.e. all of TiddlySpace) I'd expect results from all of TiddlySpace.

[chris...@gmail.com]

On Tue, 15 May 2012, Ben Gillies wrote:

> I get what you're saying, but I think there's good reason for it being
> applied as normal. That is:
>
> 1) It's applied to everything else

Yes, and in my footnote I was trying to suggest that it _not_ be. That
is, /search should come out from under ControlView.

This is not as big a change as you might imagine because:

* The server search UI exposed in the generated TiddlyWiki avoids
ControlView.
* Searches for following etc avoid ControlView.
* Other search interfaces are primarily created by you or me or by
other people who are already fiddling with Javascript to make the
search operational, and likely already have

xhr.setRequestHeader("X-ControlView", "false");

in there somewhere.
* Use of search (for not following type searches) is _low_.

> 2) If I search the bengillies space I expect results from the
> bengillies space only, unless I explicitly opt out (as per the
> X-ControlView header)
> 3) If I want to search across the whole of TiddlySpace, I can do so at
> tiddlyspace.com

I think this puts too much priority on individual spaces. One of the
areas in which I think TiddlySpace has not lived up (yet) to its
promise is that there is very little interaction/linking between
spaces. Each space is a little island, and this is very much not what
I've been working to create[1]. ControlView is a big factor, at least
mentally, blocking a more integrated infosystem.

Features that would make things more integrated include:

* a wide open search
* getting reply deployed
* enabling inter-space transclusion
* ...other stuff that has slipped my mind

> I'd say it's for people to find things without having to remember the
> syntax. If they're searching in a space, I'd expect results to be
> limited to just that space. Likewise if they search tiddlyspace.com
> (i.e. all of TiddlySpace) I'd expect results from all of TiddlySpace.

That's a UI question, not an API question. You can create search forms
that do the right thing quite easily. Right now the default is
limited, which just seems wrong from a, um, political standpoint.

You can easily imagine a search interface with radio buttons:

___ search everything
_x_ just this space
+-------------------+
|search! |
+-------------------+

I would agree with you if there were already search interfaces that
were getting lots of use, but there aren't. That lack is part of why I
put the /hsearch thing up at all: To get a search that works more like
people expect (case insensitive, searching in tags and titles along
with text, scoring based on title and tag).

[1] For me, TiddlySpace is supposed to be a tool for semi-passive
dialog. An engine around which people do compare and constrast. Not a
person. People.

It hybridizes the wiki concept by allowing individuals (and groups) to
maintain their own knowledge networks augmented, supported and refined by
other people's networks.

Much like the web at large but with a particular data model, the
tiddler.