Initial support for using AllegroCache

16 views
Skip to first unread message

John Fremlin

unread,
Feb 20, 2009, 6:11:15 AM2/20/09
to webl...@googlegroups.com
Very rough and ready.

The worst part about this is overwriting the object-id method
completely.

Suggestions and whatnot welcome -- it works for our small mock up but
we're not sure if we'll stay with weblocks (our program is mostly for
generating lots of RRD graphs -- if you want an example of doing that
with weblocks I can send it along).

(in-package #:weblocks)

(defmethod class-store (class-name) :around
(typecase (find-class class-name)
(db.allegrocache::persistent-class
db.allegrocache:*allegrocache*)
(t (call-next-method))))

(defmethod object-id (obj)
(db.allegrocache:db-object-oid obj))

(defmethod count-persistent-objects ((store db.allegrocache::database)
class-name &key &allow-other-keys)
(let ((val 0))
(let ((db.allegrocache:*allegrocache* store))
(db.allegrocache:doclass* (x (find-class class-name))
(incf val)))
val))

(defmethod find-persistent-object-by-id ((store db.allegrocache::database)
class-name id)
(db.allegrocache:oid-to-object class-name id :db store))

(defgeneric strictly-less-p (a b)
(:documentation
"Returns true if 'a' is strictly less than 'b'. This function is
used by the framework for sorting data.")
(:method (a b)
(strictly-less-p (format nil "~A" a) (format nil "~A" b)))
(:method ((a number) (b number))
(< a b))
(:method ((a string) (b string))
(not (null (string-lessp a b))))
(:method ((a null) (b null))
nil)
(:method (a (b null))
t)
(:method ((a null) b)
nil))

(defgeneric equivalentp (a b)
(:documentation
"Returns true if 'a' is in some sense equivalent to 'b'. This
function is used by the framework for sorting data.")
(:method (a b)
(equalp a b)))


(defun order-objects-in-memory (seq order-by)
"Orders objects in 'seq' according to 'order-by'."
(if (and seq
order-by)
(stable-sort seq
(if (equalp (cdr order-by) :asc)
#'strictly-less-p
(lambda (a b)
(and (not (strictly-less-p a b))
(not (equivalentp a b)))))
:key (curry-after #'slot-value-by-path (car order-by)))
seq))

;;;;;;;;;;;;;
;;; Range ;;;
;;;;;;;;;;;;;
(defun range-objects-in-memory (seq range)
"Selects only the objects in 'range' from 'seq'."
(if (and seq
range)
(let ((len (length seq)))
(subseq seq (min len (car range)) (min len (cdr range))))
seq))


(defmethod find-persistent-objects ((store db.allegrocache::database) class-name &key order-by range &allow-other-keys)
(order-objects-in-memory
(range-objects-in-memory
(let (val)
(let ((db.allegrocache:*allegrocache* store))
(db.allegrocache:doclass* (x (find-class class-name))
(push x val)))
val) range) order-by))

Leslie P. Polzer

unread,
Feb 20, 2009, 6:24:46 AM2/20/09
to webl...@googlegroups.com

> Very rough and ready.

Then it's at least a candidate for contrib/ right away.


> The worst part about this is overwriting the object-id method
> completely.

Fair enough.


> Suggestions and whatnot welcome -- it works for our small mock up

This first approximation is suprisingly simple. How do you
add new objects (or prevent new instances from getting added
automatically)?


> but we're not sure if we'll stay with weblocks

Can we help you in your decision process?

In any case I'd be glad to know with what you will end up
and what influenced your decision.


> (our program is mostly for generating lots of RRD graphs -- if you
> want an example of doing that with weblocks I can send it along).

Yes please. I have a personal interest in such a thing and it would
probably also make a nice additional demo.


Some things I noticed:

> (defgeneric strictly-less-p (a b)
> (defgeneric equivalentp (a b)
> (defun order-objects-in-memory (seq order-by)
> (defun range-objects-in-memory (seq range)

Did you copy those from the memory store?

We should really make this available for all stores
in store-utils.lisp.


> (defmethod find-persistent-objects ((store db.allegrocache::database) class-name &key
> order-by range &allow-other-keys)
> (order-objects-in-memory
> (range-objects-in-memory
> (let (val)
> (let ((db.allegrocache:*allegrocache* store))
> (db.allegrocache:doclass* (x (find-class class-name))
> (push x val)))
> val) range) order-by))

I suppose we'd want to delegate ordering and ranging to AC later
for efficiency...

Leslie

Ian Eslick

unread,
Feb 20, 2009, 7:48:25 AM2/20/09
to webl...@googlegroups.com
Weblocks makes some assumptions about persistence as object
serializability to some data store. True persistent objects don't
quite fit because in a couple of places it creates temporary objects
which are visible to the whole system, but may be uninitialized. I
had to create a proxy class mechanism (weblocks-elephant) to allow for
temporary objects that didn't pollute the class index until we were
ready to commit it.

Ian

John Fremlin

unread,
Jun 3, 2009, 12:24:13 AM6/3/09
to leslie...@gmx.net, webl...@googlegroups.com, KURODA Hisao, Jianshi HUANG
Hi Leslie,

Sorry, I didn't see your reply as I wasn't subscribed. We're now
updating the web front end for our application again, and I managed to
dig up the reply on google groups. Hopefully I'm now subscribed too.

"Leslie P. Polzer" <s...@viridian-project.de> writes:

>> Very rough and ready.
>
> Then it's at least a candidate for contrib/ right away.

You're welcome to it, for what it is.

>> The worst part about this is overwriting the object-id method
>> completely.
>
> Fair enough.
>
>
>> Suggestions and whatnot welcome -- it works for our small mock up
>
> This first approximation is suprisingly simple. How do you
> add new objects (or prevent new instances from getting added
> automatically)?

I didn't consider that, evidently ;-)

At the moment our web interface only displays things. It cannot actually
modify them. The interface is therefore correspondingly basic.

>> but we're not sure if we'll stay with weblocks
>
> Can we help you in your decision process?
>
> In any case I'd be glad to know with what you will end up
> and what influenced your decision.

Frankly the reason I chose weblocks was the awesome i-search bar.

To my horror, now we are updating the application again for more
intensive use and I wanted to put in the isearch, I discover that it is
now disabled! How can I enable i-search? Do you have an example of it
for any backend?

>> (our program is mostly for generating lots of RRD graphs -- if you
>> want an example of doing that with weblocks I can send it along).
>
> Yes please. I have a personal interest in such a thing and it would
> probably also make a nice additional demo.

Well here's the rather preliminary weblocks relevant code. I'm sure
you'd be able to figure out a much better way of doing it and I'd
appreciate any hints on a more weblocksy way to go about it.

If you're really interested I can ask my boss if we can release some of
the brains of it (the stuff in netstatus.manager is proprietary
unfortunately).

(defun graph-tmp-path ()
(merge-pathnames "tmp/" (compute-public-files-path :weblocks)))

(defvar *graph-tmp-html-path* "/pub/tmp/")

(defun clean-graph-tmp-path ()
(ensure-directories-exist (graph-tmp-path))
(loop for file in (directory (merge-pathnames "*" (graph-tmp-path)))
do (when (> (- (get-universal-time) (file-write-date file)) 300)
(delete-file file))))

(defun router-graph-show (k router)
(let ((counter 0))
(flet ((gen-filename ()
(format nil "tmp-router-~D-~D.png" (random 100000) (incf counter))))
(clean-graph-tmp-path)
(labels ((graph-router (router)
(let ((graphs (netstatus.manager::sourcegroup-graphs router)))
(when graphs
(let ((files
(loop for g in graphs collect (gen-filename))))
(loop for g in graphs
for f in files
do (netstatus.manager::generate-graph g router (merge-pathnames f (graph-tmp-path))))
(with-html
(:h2 (str (netstatus.manager::title router))))

(loop for f in files
for g in graphs do
(with-html
(:div
(:h3 (str (netstatus.manager::graph-def-title g)))
(:img :src (concatenate 'string (string *graph-tmp-html-path*) f)))))
(typecase router
(netstatus.manager::router
(loop for if in (netstatus.manager::router-interfaces router) do
(graph-router if)))))))))
(graph-router router))))

(render-link
(lambda(&rest args)
(declare (ignore args))
(answer k)) "Back"))

(defun router-graph (obj router)
(declare (ignore obj))
(do-page
(lambda(k)(router-graph-show k router))))


(defun make-routers-page ()
(make-instance 'composite :widgets
(list
(make-instance 'datagrid
:on-drilldown '(graph . router-graph)
:name 'routers-grid
:data-class 'router
:view 'router-table-view
:item-data-view 'router-data-view
:item-form-view 'router-form-view))))


>
>
> Some things I noticed:
>
>> (defgeneric strictly-less-p (a b)
>> (defgeneric equivalentp (a b)
>> (defun order-objects-in-memory (seq order-by)
>> (defun range-objects-in-memory (seq range)
>
> Did you copy those from the memory store?

Yes.

> We should really make this available for all stores
> in store-utils.lisp.

Yes, that would definitely be a good idea. I notice the elephant
backend just ::'s the symbols from weblocks-memory.

>> (defmethod find-persistent-objects ((store db.allegrocache::database) class-name &key
>> order-by range &allow-other-keys)
>> (order-objects-in-memory
>> (range-objects-in-memory
>> (let (val)
>> (let ((db.allegrocache:*allegrocache* store))
>> (db.allegrocache:doclass* (x (find-class class-name))
>> (push x val)))
>> val) range) order-by))
>
> I suppose we'd want to delegate ordering and ranging to AC later
> for efficiency...

It turns out that the network AllegroCache wasn't very suitable for the
usage we need (frequent modifications involving all objects in the
database), and we've been using anardb.

http://cl-www.msi.co.jp/projects/anardb/index.html

I basically copied the old code I had with allegrocache for the database
link.


anardb.lisp

Leslie P. Polzer

unread,
Jun 3, 2009, 3:03:02 AM6/3/09
to webl...@googlegroups.com

John Fremlin wrote:

>> Then it's at least a candidate for contrib/ right away.
>
> You're welcome to it, for what it is.

Alright, I'm going to put it in.


> Frankly the reason I chose weblocks was the awesome i-search bar.
>
> To my horror, now we are updating the application again for more
> intensive use and I wanted to put in the isearch, I discover that it is
> now disabled! How can I enable i-search? Do you have an example of it
> for any backend?

This feature seems to be really popular, and a lot of people
miss it. I don't know why it vanished and what exactly it would
take to get it back.


> Well here's the rather preliminary weblocks relevant code. I'm sure
> you'd be able to figure out a much better way of doing it and I'd
> appreciate any hints on a more weblocksy way to go about it.

There's not much Weblocks in it so just two more or less trivial
hints:


> (with-html
> (:h2 (str (netstatus.manager::title router))))

You always want to use ESC instead of STR for strings that make
it to the final HTML, except if they contain HTML themselves
(and then they must in turn escape their content properly).


> (render-link
> (lambda(&rest args)
> (declare (ignore args))
> (answer k)) "Back"))

You can just use F_% from f-underscore in place of the lambda,
its arglist and its ignore declaration.

By the way that (loop for if in list ...) confused me somewhat.
I didn't really expect a part of the LOOP language to be used
as a variable.


> It turns out that the network AllegroCache wasn't very suitable for the
> usage we need (frequent modifications involving all objects in the
> database), and we've been using anardb.
>
> http://cl-www.msi.co.jp/projects/anardb/index.html
>
> I basically copied the old code I had with allegrocache for the database
> link.

That's an interesting project that escaped my attention, thanks!

Leslie

nunb

unread,
Jun 4, 2009, 10:56:39 AM6/4/09
to weblocks
> Frankly the reason I chose weblocks was the awesome i-search bar.
>
> To my horror, now we are updating the application again for more
> intensive use and I wanted to put in the isearch, I discover that it is
> now disabled! How can I enable i-search? Do you have an example of it
> for any backend?

AFAIK, making the search bar work sensibly across backends was
difficult. I have been using a version of incremental search, a paste
is attached and it should be in contrib soon. Let me know if it
addresses your use-case?

The idea is the search works only for the data which is shown in the
table, and it is case-insensitive.

http://paste.lisp.org/display/81353



cheers,
nb

John Fremlin

unread,
Aug 7, 2009, 12:55:57 AM8/7/09
to weblocks
nunb <nandan.bagc...@gmail.com> writes:
[...]

> AFAIK, making the search bar work sensibly across backends was
> difficult. I have been using a version of incremental search, a paste
> is attached and it should be in contrib soon. Let me know if it
> addresses your use-case?
>
> The idea is the search works only for the data which is shown in the
> table, and it is case-insensitive.
>
> http://paste.lisp.org/display/81353

Thanks for the link. That looks like a slightly different approach to
the old and resurrected isearch.

What happened to it? Has it made it into the contrib? Is there an
example demonstrating what it looks like so I can try it out before
integrating it into our app?

(PS. I'm sorry for being dense but I'm not sure how to attach the
resurrected standard isearch to a grid -- is there any example
demonstrating the use of it?)


[...]

Leslie P. Polzer

unread,
Aug 7, 2009, 3:54:43 AM8/7/09
to webl...@googlegroups.com

John Fremlin wrote:

> Thanks for the link. That looks like a slightly different approach to
> the old and resurrected isearch.

It needs to. The old isearch vanished when store abstractions were
introduced because it's not easy to provide efficient isearch across
different stores and views.


> What happened to it?

It's waiting for someone to say what should happen to it. :)


> Has it made it into the contrib?

No.


> Is there an example demonstrating what it looks like so I can try it out
> before integrating it into our app?

nunb might be able to provide a screenshot or two.


> (PS. I'm sorry for being dense but I'm not sure how to attach the
> resurrected standard isearch to a grid -- is there any example
> demonstrating the use of it?)

You probably need to let your grid inherit from FILTERED-GRID,
ensure that the methods are compatible with your grid's methods
(if any) and replace occurences of FOOX-GRID in the method lambda
lists with FILTERED-GRID.

Leslie

Maciej Katafiasz

unread,
Aug 7, 2009, 12:57:27 PM8/7/09
to weblocks


On Aug 7, 9:54 am, "Leslie P. Polzer" <s...@viridian-project.de>
wrote:
> John Fremlin wrote:
> > Thanks for the link. That looks like a slightly different approach to
> > the old and resurrected isearch.
>
> It needs to. The old isearch vanished when store abstractions were
> introduced because it's not easy to provide efficient isearch across
> different stores and views.

What happened to the flag specifying if the store supports full-text
searches? It used to be there.

Also, I'm beginning to believe that the store abstraction needs to be
a lot more sophisticated in what it provides querying-wise, we used to
run into mismatches that'd result in turning SQL-able queries into
looping over results in the Lisp code all the time in the cl-perec
backend (which is, before you ask, scrapped for the time being, due to
it making us fight on two fronts and consequently progressing on none.
We might resurrect it when the logic is correct, and when we have a
reasonable idea of how we want it to operate, exactly).

Perec has a very effective query compiler that can turn queries
expressed in terms of lisp types and funcalls into real SQL, but
because there's hardly any query interface to stores, there was no way
to pass that info down. I know that a "generic ORM layer" is a doomed
project, but what we have right now is the "generic client-side
looping over SQL results". I think a store-agnostic query compiler
would be in order (maybe something could be stolen from Perec
itself?), then simple stores could continue to be simple and function
with the Lisp-side processing, but real DB interfaces could implement
a more sophisticated language, as they have the means to support it.

Cheers,
Maciej

Leslie P. Polzer

unread,
Aug 7, 2009, 3:49:15 PM8/7/09
to weblocks
On Aug 7, 6:57 pm, Maciej Katafiasz <mathr...@gmail.com> wrote:

> What happened to the flag specifying if the store supports full-text
> searches? It used to be there.

I can't remember such a flag, but that doesn't mean it hasn't been
there at some time...


>  Also, I'm beginning to believe that the store abstraction needs to be
> a lot more sophisticated in what it provides querying-wise, we used to
> run into mismatches that'd result in turning SQL-able queries into
> looping over results in the Lisp code all the time in the cl-perec
> backend (which is, before you ask, scrapped for the time being, due to
> it making us fight on two fronts and consequently progressing on none.
> We might resurrect it when the logic is correct, and when we have a
> reasonable idea of how we want it to operate, exactly).

Yes, the store abstraction level needs to be refined. Besides your
Perec backend a query interface would also benefit the Elephant
and SQL stores.


> I think a store-agnostic query compiler
> would be in order (maybe something could be stolen from Perec
> itself?), then simple stores could continue to be simple and function
> with the Lisp-side processing, but real DB interfaces could implement
> a more sophisticated language, as they have the means to support it.

A very interesting idea, let's keep it in mind until everyone has a
bit more
time on their hands. I'd like to encourage you to file a ticket on the
Bitbucket
tracker so we don't forget.

Leslie

John Fremlin

unread,
Aug 13, 2009, 4:27:31 AM8/13/09
to webl...@googlegroups.com, KURODA Hisao, Jianshi HUANG
The isearch feature is IMHO one of the most awesome aspects of
weblocks. Even before an efficient interface to the underlying store is
made, it would be really nice to have it.

Here's a version of nunb's great filtered grid
(http://paste.lisp.org/display/81353) which (a) compiles, (b) doesn't
depend on sbcl, and (c) moves the isearch box to the top of the grid.

A lot of other changes were made.

What do you think about it? Perhaps it should be a mixin instead as
nunb suggested? I think MSI might be willing to do a bit of work to get
it tidy enough to be included.

I guess there is this worrying comment
;(send-script (ps* `(.activate ($ ,sym)))) ;why the hell does this not work?
what is the webloxy way of doing focus?

(in-package :weblocks)

(defwidget filtered-grid (gridedit)
((filters :initform nil :accessor grid-filters :documentation "a regex string to search for")))

(defmethod dataseq-render-pagination-widget ((obj filtered-grid) &rest args)
(with-html
(:div :class "pagination-side"
(dataseq-render-pagination-widget-default obj args))))

(defmethod render-widget-body :before ((obj filtered-grid) &rest args)
(with-html
(:div :class "search-side" :id "search_ui_id"
(fg-render-search-ui obj))))

(defmethod render-dataseq-body ((obj filtered-grid) &rest args)
(let* ((data-sequence (dataseq-data obj)))
(when (and (grid-filters obj) (text-input-present-p (grid-filters obj)))
(setf data-sequence
(fg-filter-sequence obj data-sequence)))
(setf (slot-value obj 'rendered-data-sequence) nil) ;Can this be used to cache? Why is it here (did it have something to do with the old isearch?)
(with-html
(:div :class "datagrid-body"
(apply #'render-object-view data-sequence (dataseq-view obj)
:widget obj
:summary (if (dataseq-sort obj)
(format nil "Ordered by ~A, ~A."
(string-downcase
(humanize-name
(dataseq-sort-slot obj)))
(string-downcase
(humanize-sort-direction
(dataseq-sort-direction obj))))
nil)
:custom-fields (append-custom-fields
(remove nil
(list
(when (dataseq-allow-select-p obj)
(cons 0 (weblocks::make-select-field obj)))
(when (and (dataseq-allow-drilldown-p obj)
(dataseq-on-drilldown obj))
(weblocks::make-drilldown-field obj))))
args)
args))
(setf (slot-value obj 'rendered-data-sequence) data-sequence))))

(defmacro closure (&body body)
(with-unique-names (args)
`(lambda (&rest ,args)
(declare (ignore ,args))
,@body)))

(defmethod fg-render-reset ((obj filtered-grid))
(with-html
(:span :class "reset"
(render-link (closure (fg-reset-filters obj)) "reset"))))

(defmethod fg-reset-filters ((obj filtered-grid))
(setf (grid-filters obj) nil)
(mark-dirty obj))

(defmethod fg-render-search-ui ((obj filtered-grid) &rest args)
(let ((sym (gensym "search")))
(render-isearch "filter" (f (&rest args &key filter &allow-other-keys)
(setf (grid-filters obj) filter)
(mark-dirty obj))
:class (if (grid-filters obj) "grid_search filtered" "grid_search empty")
:value (grid-filters obj)
:input-id sym)

;(send-script (ps* `(.activate ($ ,sym)))) ;why the hell does this not work?
(send-script (format nil "$('~A').focus();" sym))
))


(defmethod fg-filter-sequence ((obj filtered-grid) seq)
(let* ((filters (grid-filters obj))
(scanner (cl-ppcre:create-scanner filters :case-insensitive-mode (notany #'upper-case-p filters))))
(with-standard-io-syntax
(remove-if-not
(lambda (item)
(let ((obj-as-string
(format nil "~{~A~}" (fg-render-object-view-body-row (dataseq-view obj) item obj))))
(cl-ppcre:all-matches scanner obj-as-string)))
seq))))

;;; Getting a table-view like view of this thingie (in fg-filter-sequence)
;;; instead of (format nil "~{~A~}" (loop for slot in slots collect (slot-value item (sb-mop:slot-definition-name slot))))
(defmethod fg-render-object-view-body-row (view obj widget &rest args)
"Render a string representation of a data object for full-text search"
(apply #'map-view-fields
(lambda (field-info)
(let ((field (field-info-field field-info))
(obj (field-info-object field-info)))
(apply #'fg-render-view-field
field view widget (view-field-presentation field)
(obtain-view-field-value field obj) obj
args)))
view obj args))

(defmethod fg-render-view-field (field view
widget presentation value obj
&rest args)
(apply #'print-view-field-value value presentation field view widget obj args)) ; or use #'render-view-field-value?


[...]


Leslie P. Polzer

unread,
Aug 13, 2009, 5:07:57 AM8/13/09
to weblocks
On Aug 13, 10:27 am, John Fremlin <j...@msi.co.jp> wrote:
> The isearch feature is IMHO one of the most awesome aspects of
> weblocks. Even before an efficient interface to the underlying store is
> made, it would be really nice to have it.

Yes, indeed. It's a great selling point and really convenient, too.

But it's worrying that it will produce wrong results when used
with a store that actually heeds pagination.

Assume we're showing items 1-10 gathered from the CLSQL store;
then we will never filter the rest of the items at all (because
DATASEQ-DATA
will only work on the paginated set). This is not how the original
isearch
worked and messes with your expectations.

So: efficiency considerations can wait, but we need to ensure that
we're
delivering correct results.

Hacking a custom version of DATASEQ-DATA (preferably while duplicating
as little as possible) for filtering purposes would help.


> What do you think about it?  Perhaps it should be a mixin instead as
> nunb suggested?

If that is feasible, yes.


> I think MSI might be willing to do a bit of work to get it tidy enough to be
> included.

That would be great. The current version would need some minor
beautification, a fix for the problem above and ideally some tests.


> I guess there is this worrying comment
> ;(send-script (ps* `(.activate ($ ,sym)))) ;why the hell does this not work?
> what is the webloxy way of doing focus?

Right now there is none. With Mackram's Javascript abstraction
focussing
is probably part of the common JS API, though.

Leslie

John Fremlin

unread,
Aug 14, 2009, 5:20:33 AM8/14/09
to webl...@googlegroups.com, KURODA Hisao
Hi Leslie,

"Leslie P. Polzer" <leslie...@gmx.net> writes:
[...]


> Assume we're showing items 1-10 gathered from the CLSQL store;
> then we will never filter the rest of the items at all (because
> DATASEQ-DATA
> will only work on the paginated set). This is not how the original
> isearch
> worked and messes with your expectations.

Thanks for your explanation.

I've made a completely rewritten version taking this into account.

Sorry for the code dump, maybe I will have some time to look at it next
week.

weblocks-dump.diff

Leslie P. Polzer

unread,
Aug 14, 2009, 11:39:05 AM8/14/09
to weblocks
On Aug 14, 11:20 am, John Fremlin <j...@msi.co.jp> wrote:

> One major issue is that if isearch box is marked dirty then when the
> ajax call comes back, it will overwrite any further user input. This is
> extremely annoying with slow AJAX.
>
> I would like to fix this before tidying up. Maybe the mixin approach is wrong?

I don't think the mixin approach is responsible for this.

We rather seem to need a piece of custom Javascript to deal with this.
Proposal after the AJAX request has completed:

1. protect the text field against further modification
2. save the current contents of the text field
3. take the response of the AJAX request and apply it
4. restore the previously saved contents

John Fremlin

unread,
Aug 17, 2009, 3:50:54 AM8/17/09
to webl...@googlegroups.com, Jianshi HUANG, KURODA Hisao
"Leslie P. Polzer" <leslie...@gmx.net> writes:
> On Aug 14, 11:20 am, John Fremlin <j...@msi.co.jp> wrote:
>
>> One major issue is that if isearch box is marked dirty then when the
>> ajax call comes back, it will overwrite any further user input. This is
>> extremely annoying with slow AJAX.
>>
>> I would like to fix this before tidying up. Maybe the mixin approach is wrong?
>
> I don't think the mixin approach is responsible for this.
>
> We rather seem to need a piece of custom Javascript to deal with this.
> Proposal after the AJAX request has completed:

Am I correct in thinking that only dirty objects are updated by the AJAX
request? (If not that is a shame.) If so, then the trick is to make sure
that the filter input box is not marked dirty.

The solution as I see it is to make a parent widget, attach the dataseq
and the filter as children of this widget, and mark on the dataseq dirty
when the filter changes.

Therefore the mixin approach is out, and a container approach is in.

> 1. protect the text field against further modification

That sounds rather irritating for the poor user.

> 2. save the current contents of the text field
> 3. take the response of the AJAX request and apply it
> 4. restore the previously saved contents

I'd rather use the container approach instead. However, if for some
reason that were not feasible, and magic JavaScript is needed, I'd
rather say --

(1) Do the AJAX request.

(2) On receipt of the AJAX answer, if the isearch contents have
changed, then do not apply but discard the AJAX answer (and potentially
make a new request).

[...]

Leslie P. Polzer

unread,
Aug 17, 2009, 4:02:40 AM8/17/09
to webl...@googlegroups.com

John Fremlin wrote:
>
> "Leslie P. Polzer" <leslie...@gmx.net> writes:
>> On Aug 14, 11:20 am, John Fremlin <j...@msi.co.jp> wrote:
>>
>>> One major issue is that if isearch box is marked dirty then when the
>>> ajax call comes back, it will overwrite any further user input. This is
>>> extremely annoying with slow AJAX.
>>>
>>> I would like to fix this before tidying up. Maybe the mixin approach is wrong?
>>
>> I don't think the mixin approach is responsible for this.
>>
>> We rather seem to need a piece of custom Javascript to deal with this.
>> Proposal after the AJAX request has completed:
>
> Am I correct in thinking that only dirty objects are updated by the AJAX
> request?

That's right. Note that there's a basic dependency system for
the dirty mechanism.


> (If not that is a shame.) If so, then the trick is to make sure
> that the filter input box is not marked dirty.

My first idea, too. But I think the original isearch also gave
feedback to the user, turning red when the search string couldn't
be found.

The only solution I see here is smart client-side code.


>> 1. protect the text field against further modification
>
> That sounds rather irritating for the poor user.

I don't think so. We're talking about a time span that will
hardly be noticeable at all.


> (2) On receipt of the AJAX answer, if the isearch contents have
> changed, then do not apply but discard the AJAX answer (and potentially
> make a new request).

That would be another acceptable solution I guess.

Leslie

nunb

unread,
Aug 18, 2009, 4:32:10 AM8/18/09
to weblocks
> > Assume we're showing items 1-10 gathered from the CLSQL store;
> > then we will never filter the rest of the items at all (because
> > DATASEQ-DATA
> > will only work on the paginated set). This is not how the original
> > isearch
> > worked and messes with your expectations.
>
> Thanks for your explanation.
>
> I've made a completely rewritten version taking this into account.

I recently ran into this as our dataset grew, so thanks for doing
this!


> > That would be great. The current version would need some minor
> > beautification, a fix for the problem above and ideally some tests.
>
> One major issue is that if isearch box is marked dirty then when the
> ajax call comes back, it will overwrite any further user input. This is
> extremely annoying with slow AJAX.
>

Yes, we had this issue too. Since ours is an internal app the problem
wasn't too severe. The UI person actually insisted that for most
users, they would expect to press enter before the ajax went off and
did it's thing.

A version that does things this way is at http://paste.lisp.org/+1U15
with supporting annotations (but note it suffers from the earlier
problem of paginated data sets not being properly searched).

I haven't looked at your code yet, but fwiw, in our case moving the
searchbox to the top was handled by css absolute positioning.

Caveat: we've made a decision to go all desktoppy and so most of our
code requires JS. We also require JS because we're trying to make our
forms more dynamic and have a pressing need to have form elements
interact with each other. So if degradability is important the old
render-isearch with the Prototype timeout/delay would be more
suitable.

> I would like to fix this before tidying up. Maybe the mixin approach is wrong?

I had intended for this to be fairly easy to derive from if search
functionality was required; and to be ignored if not. But considering
the other issues that have been brought up I think it should be folded
into the gridedit hierarchy somewhere once the stores abstraction
takes shape. I still think that could be left to an advanced search
and the simple search should be almost like emacs C-s or vim /-search.



nunb

unread,
Aug 18, 2009, 4:51:06 AM8/18/09
to weblocks
> >     (2) On receipt of the AJAX answer, if the isearch contents
have
> > changed, then do not apply but discard the AJAX answer (and potentially
> > make a new request).

Afaik, this would be rather hard to do since weblocks doesn't give us
hooks into the Ajax-JS processing [1]

Instead, the dirty objects are marked and collected Lisp-side, the
json hash of {widgetid --mapped--> new-html} is created, and then in
one fell swoop in weblocks.js, their divs are updated with the new
html.

Another, more promising table approach was using the YUI table and
writing a RESTy search function that performed the search and returned
JSON data which YUI knew how to display. Being more JS oriented it was
also easier to do table-size animations and suchlike, iirc. I used the
basic table example from http://developer.yahoo.com/yui/examples/datatable/dt_autocomplete.html

The following cool code from Leslie lets us selectively exempt
functions from regular AJAX processing (http://paste.lisp.org/+1U15#4)
-- please ignore the crufty hellacious comments if any ;-)

hth,
Nandan

[1] (I've had some use-cases where these would be welcome, but they
were all related to UI-tweaking which can wait).

nunb

unread,
Aug 18, 2009, 6:00:56 AM8/18/09
to weblocks
> A version that does things this way is athttp://paste.lisp.org/+1U15
> .....

I should have mentioned that it uses shortcut.js from
http://www.openjs.com/scripts/events/keyboard_shortcuts/

John Fremlin

unread,
Aug 21, 2009, 3:50:50 AM8/21/09
to webl...@googlegroups.com
nunb <nandan....@gmail.com> writes:
[...]

> The following cool code from Leslie lets us selectively exempt
> functions from regular AJAX processing (http://paste.lisp.org/+1U15#4)
> -- please ignore the crufty hellacious comments if any ;-)

Aaargh. Kind of defeats the point of using weblocks at all :-)

[...]

John Fremlin

unread,
Aug 21, 2009, 3:53:01 AM8/21/09
to webl...@googlegroups.com
nunb <nandan....@gmail.com> writes:
[...]

>> One major issue is that if isearch box is marked dirty then when the
>> ajax call comes back, it will overwrite any further user input. This is
>> extremely annoying with slow AJAX.
>>
>
> Yes, we had this issue too. Since ours is an internal app the problem
> wasn't too severe. The UI person actually insisted that for most
> users, they would expect to press enter before the ajax went off and
> did it's thing.

That makes sense.

> A version that does things this way is at http://paste.lisp.org/+1U15
> with supporting annotations (but note it suffers from the earlier
> problem of paginated data sets not being properly searched).
>
> I haven't looked at your code yet, but fwiw, in our case moving the
> searchbox to the top was handled by css absolute positioning.

Hehe, a bit too web development-licious for me ;-)

[...]

>> I would like to fix this before tidying up. Maybe the mixin approach is wrong?
>
> I had intended for this to be fairly easy to derive from if search
> functionality was required; and to be ignored if not. But considering
> the other issues that have been brought up I think it should be folded
> into the gridedit hierarchy somewhere once the stores abstraction
> takes shape. I still think that could be left to an advanced search
> and the simple search should be almost like emacs C-s or vim /-search.

At the moment my version works well as a mixin except for this dirtiness
issue.

Did the original isearch have this dirtiness thing fixed? I guess we
should really just revert our weblocks to the latest non-borked version,
as the isearch is the main feature we want.

[...]

Leslie P. Polzer

unread,
Aug 22, 2009, 5:27:27 AM8/22/09
to weblocks
On Aug 21, 9:50 am, John Fremlin <j...@msi.co.jp> wrote:

> Aaargh. Kind of defeats the point of using weblocks at all :-)

Not really. Like jumping the stack (in fact it's based on THROW) it's
useful
in certain situations.

Leslie P. Polzer

unread,
Aug 22, 2009, 5:29:45 AM8/22/09
to weblocks
On Aug 21, 9:53 am, John Fremlin <j...@msi.co.jp> wrote:

> Did the original isearch have this dirtiness thing fixed?

I'm not sure. You can try it by checking out the appropriate revision.
I think it's using custom JS in any case.


> I guess we should really just revert our weblocks to the latest
> non-borked version, as the isearch is the main feature we want.

I don't get this -- what is broken and where do you want to revert
to? :)

Leslie
Reply all
Reply to author
Forward
0 new messages