Yes, the idea is to have several backends like this
pub/scripts/backends
/prototype
/jquery
/yui
Each one of those would have a matching DEFINE-JAVASCRIPT-BACKEND
expression.
Each webapp could then select one of those by its unique name.
Example:
(defwebapp foo :prefix "/" :javascript-backend prototype)
> Now my problem is trying to
> figure out where weblocks adds all its js code. A bit of search lead
> me to application.lisp where the different js files are appended.
> However the demo outputs a combined file
Note: you can turn this off by specifying :DEBUG T.
> .... how does it do it and where does the aggregator, so to speak,
> gets its list of files?
The list of files is fed into COMPACT-DEPENDENCIES which in turn
calls BUNDLE-DEPENDENCIES. See bundling.lisp and versioning.lisp
for the whole fun.
But I don't think you need to understand bundling. You just
need to parameterize the list of default application dependencies
(in application.lisp) on the selected js backend.
Leslie
> So i will work on porting the js files to jquery and will implement
> the functionality in this weekend. By that way hopefully ver 0.8.3
> will have support out of the box for both jquery and prototype :)
Cool! Thanks in advance for your work!
It's about time we got rid of our Prototype lock-in.
> basically i am asking why not port shortcut (I can understand datagrid
> would be called by its respective widget and hence does not explicitly
> get called in applicaiton).
Shortcut isn't tied to a particular backend, it's self-sufficient.
And a small note: we (or initially you, whichever you prefer)
need to define an abstract API for the js backend that needs
to be implemented by each one.
Example:
/* abstract as used by Weblocks files */
function applyEffect(type, targetEl) {}
/* prototype/scriptaculous impl */
function applyEffect(type, targetEl) {
switch (type) {
case 'fade':
Effect.fade(target);
break
// ...
}
/* jquery impl */
[...]
That should also resolve your question about datagrid.js
and dialog.js: they may remain in the standard application
dependencies and need to be converted to use the abstract
API.
Yes, that was my intention above, I just left out the file names for
brevity.
However I think that every backend should have a separate
directory to make it possible to use multiple files without
cluttering one top-level directory.
> Basically what i am suggesting is that rathere than having the (define-
> javascript-backend NAME ROOT FILES) as mentioned in the docs we would
> have (define-client-backend NAME :js-root JROOT :js-files JFILES :css-
> root CROOT :css-files CFILES). The definition of paths will follow
> that of the doc.
>
> I think this would give us a better step for the future, but I want to
> make sure it is inline with what we want to be added to the weblocks
> main branch so what do you think?
Looks good to me!
Those two changes are perfectly alright.
> 3- I have had only 3 failures in all the tests and they are all under
> handle-request.lisp. Specifically I am having trouble with handle-
> client-request-6, handle-client-request-7 (in both these cases I get
> result 1 when 2 is expected) and handle-client-request-10 (I get the
> result "nil" when the result expected is "http://nil/?weblocks-
> session=1%3atest"). My question is can anyone point me on to what
> these are trying to test specifically?
It's unfortunate that the tests are just numbered instead
of having a descriptive name.
I've glossed over them:
HCR-6 tests PRE-RENDER and POST-RENDER session hooks.
HCR-7 tests PRE-ACTION and POST-RENDER request hooks.
HCR-10 tests PRE-ACTION and POST-ACTION session hooks and
simple action handling.
For 6 and 7 (and maybe the second value of 10 too) you
need to check why the hooks don't get run.
For 10 check where redirects happen (or where they don't
but should). Also check manually whether a request of
the form http://server/?action=foo causes a redirect.
For each widget that is rendered we check whether there exists
a CSS and JS file with its class name (i.e. datagrid.js as Javascript
for DATAGRID). If it exists we include it.
> On a side note if I force the abstraction to show datagrid-
> prototype.js all the time then we are done but I do not like this
> approach so help is needed.
Widgets should be backend-agnostic; they should only use
functions from the common abstract JS API.
Example:
API (e.g. pub/scripts/backend/weblocks-api/foo.js):
weblocks.doFoo() = function ()
{ throw "Uh oh, doFoo() not defined for your backend. :("; };
Backend1 (e.g. pub/scripts/backend/prototype/foo.js):
weblocks.doFoo = function () { prototype.doFoo(); };
Backend2 (e.g. pub/scripts/backend/jquery/foo.js):
weblocks.doFoo = function () { jquery.doFoo(); };
Caller (e.g. datagrid.js in pub/scripts):
weblocks.doFoo(); /* must NOT use or know about
prototype.doFoo or jquery.doFoo */
> And oh the abstraction passes all the tests so we are almost
> done.
Ah, so you've managed to fix those test cases we talked about?
That's good.
Leslie
The code responsible for sending deps on AJAX requests is
in RENDER-DEPENDENCY-IN-AJAX-RESPONSE, which is called
by the default RENDER-WIDGET method (src/widgets/widget/widget.lisp).
We just send all deps for widgets marked dirty in the AJAX
request to ensure that the client has them.
This is crude since it doesn't take into account dependencies
that are already existing. Plus it doesn't work in all cases
(<script src="..."/> comes to mind, and there are other
subtleties), but all in all it does more good than harm,
especially as far as CSS is concerned.
But feel free to propose a better mechanism.
I think that the penalty isn't that high since CSS and JS files
should always be cached anyway.
But it would be much cleaner with your proposal, plus a bit more efficient
especially when one does have external files without cache time.
So it would be cool indeed if you enhanced this mechanism.
Leslie
I'm likewise filled with great pleasure to hear this!
> I still have a few things such as changing the other examples and also
> modifying the default application not to mention writing the jquery
> part because I want to use it :). But in any case this list is pretty
> short and would probably be done in a few hours. So if it is not too
> stupid a question how do we go about merging my change with the main
> code branch?
I've noticed that you already forked a repository on Bitbucket.
Here are the next steps:
1. push your code to your public repository
2. notify us and wait for comments
3. fix missing things or things that need to be corrected.
go back to (2) as necessary.
After that I'll pull your changes into -dev from where they
will eventually make their way into -stable.
> Luckily that was an easy fix ;). Anyways, if you happen to know of
> anything inside weblocks that is in need of immediate work to make it
> more production ready, please let me know I am willing to volunteer
> the time after all it will just make my work even better :).
I like your attitude. Now there's one thing that would be suitable
for immediate work and will provide great gains, and that is the
view system rework.
You can find a lot on this topic by searching the discussion archive
for 'views'. In short there are two big problems:
1. Views and widgets share a common problem domain but are quite
separate right now. The limited view DSL is nice for a lot of
purposes but often you need to fully replace it with a widget
or resort to ugly hacks to get the flexibility you need.
2. Views (and presentations, and parsers) are singletons. This
makes it difficult to dynamically configure views per-request
and even per-session.
Both of these problems can be tackled at once by implementing views
via widgets. Jan Rychter and Saikat Chakrabarti had some interesting
ideas.
In fact some of the work has already been done by Saikat.
You can find it here:
http://bitbucket.org/saikat/weblocks-saikat/
Leslie
great, I already thought I'd turned you off with my comments, heh.
>> * please try to keep your lines (including docstrings) roughly to 80
>> characters.
> fair enough. Any other coding standards we are following?
We're pretty loose about that right now.
I have (and I believe Stephen has too) found it helpful for
diffability/mergeability to have one symbol per line in EXPORT
expressions.
No other things come to my mind right now.
>> * define-client-backend: not sure if this should be a DEFVAR instead
>> of a DEFPARAMETER...
> Well I was thinking I wanted to overwrite the same symbol name so in
> case we provide some symbol say jquery by default the user would be
> able to change it by simply calling define-client-backend with symbol
> jquery later on while if it was DEFVAR then he is stuck with the
> default setup.
Let's keep it that way then.
>> * build-client-backend: let's use NCONC here instead of APPEND.
>> This dep stuff is in the performance-critical path.
> Thanks for the tip did not know that nconc had a shorter execution
> then APPEND.
Of course don't forget that the original list is clobbered with
NCONC. In the cases I mentioned we know that we're operating on
a private list so it's alright.
> (by the way any suggestions how i can profile/debug
> weblocks while i am running it a service ie with hunchentoot serving
> the pages)
You're lucky, I just implemented that very recently.
Try searching the group for "timings" and let me know
if you have any questions.
>> * test HCR-10 (I think): prototype isn't quoted. This test
>> will fail.
> I believe you were looking at not the final commit since the final
> commit I uploaded had passed all the tests.
No worries, I will test the whole thing after the pull anyway.
> Sure I will abstract that to one function. However there are no
> effects that are in the core file that we have (I think) and all
> should be implemented as I understood so correct me if I am wrong.
I'm not sure what you mean here. The Flash widget for example
uses the Appear effect of Scriptaculous.
Leslie
You've asked for two things: (1) what the browser asks for
(2) what the answer contains.
The above answers (2).
As for (1): in Hunchentoot 1.0 it doesn't work as neatly
as in 0.15 because PROCESS-REQUEST just gets a request
object. Use this instead:
(defmethod weblocks:handle-client-request :before (app)
(format t "uri: ~S~%" (hunchentoot:request-uri*)))
Leslie
> Not to stress you out or anything, but any progress? I'd like to
> avoid writing a huge bunch of code for prototype only to port it to
> jquery later on when it becomes available.
I will try to review the code until next weekend.
You can rely on this code making it into mainline in any case though,
so do write your code for JQuery.
> Prototype is very much not my favourite framework
Yes. To be honest I'm not sure why it still exists except
for backwards compatibility.
> (I had a brief professional encounter with Rails, y'see, and it
> left a distaste for both),
Not to carry this too far OT here, but I'm curious what you
didn't like about it?
> Also, I have thoughts about the JS in general, but that's
> blocking on me writing the huge email I mentioned in another thread.
Please let us hear about it when you're ready.
Leslie
> 1- I only defined jquery because I though you said you want it to be
> the default one. Defining prototype is easy (I already tested it with
> both jquery and prototype) and I can put it in tonight when i get home
> (different code machines).
Well my main question is why you define the backend in the Weblocks
demo code. Shouldn't it be part of the core code?
> As for what happens, well since we define
> the symbol as a defparameter it will get overwritten which is i think
> how it should be done. The reason is that if someone wants to add even
> more functionality to the definition they could and are not stuck with
> what we define.
Yes, that sounds sensible. I seem to remember that we discussed this
topic already.
> 2- For the namespace i think that is a good idea (i thought about it
> but i can not seem to remeber why i did not do it) I can easily do the
> change if you want although I would not go so far as specialized APIs
> since as i mentioned previously all "weblocks" API should be generic
> and each person will extend in his own library the way they want. Tell
> me what you think and if you want i can do it quickly.
It seems to be enough to put everything under the "Weblocks"
(or "weblocks") namespace to start with.
Leslie
> The issues seem pretty simple so I will work and upload in the coming
> two or three days (having some connection issues in my new "office") I
> also finished the change so that all the javascript are in the
> weblocks namespace.
Great! Don't forget to merge your branch with -dev afterwards that so
I can pull without conflicts. Your work should be merged soon after
that.
Leslie
> The code does have effect since it stops any previous loaded js from
> loading again. Also my testing showed that page deps are being sent in
> the ajax request.
I know what you're trying to do here, but I doubt that *PAGE-DEPENDENCIES*
has any effect. Maybe this helps sort it out:
This variable is bound in handle-client-request and used in
render-widget. But we only modify it in this place when we're
not in an AJAX request.
So how could it be anything else than NIL when an AJAX request
is happening?
> On a different note I did not merge with dev. I tried to do the merge
> but when I did I ended up with 78 errors in the test suite so I am not
> sure if I am doing something wrong or what can you help since the
> errors had nothing to do with the javascript changes (about 20 were
> navigation related)
It must be some changeset of yours since the current -dev test cleanly
apart from one case which involves those pesky versioning bits. :)
Try loading TRIVIAL-SHELL before you load the test suite, it will
point out the differences between expected and actual test results.
Bisecting your changesets is another effective option.
Leslie
the error is here:
(defun make-local-dependency (type file-name &key do-not-probe media
(webapp (current-webapp)) (import-p nil) (backend-root ""))
"Make a local (e.g. residing on the same web server) dependency of
type :stylesheet or :script. Unless :do-not-probe is set, checks if
file-name exists in the server's public files directory, and if it does,
returns a dependency object."
(let* ((relative-path (public-file-relative-path type
(merge-pathnames file-name
(if (string/= backend-root "")
(maybe-add-trailing-slash backend-root)
""))))
You're calling public-file-relative-path with a pathname but it's not prepared
for that. This function needs to be fixed.
Leslie
> Can you please recheck and let me know? (PS I faced this problem
> before and I fixed it by deleting all the .fasl files in both test and
> src with find and then compiled and loaded the whole thing again can
> you try that).
I've made sure I'm up to date and deleted all FASLs. One problem
remains:
ERROR : widgets/dataedit/dataedit-suite : create-new-item-widget-gridedit-1
Condition: error opening
#P"/home/sky/projects/lisp/weblocks.mackram/pub/stylesheets/vzn/dataform-import.0.css":
I'm sure this can be fixed easily...
I get this when I check out the current versions of -dev and your
branch, then try to pull and merge your branch into -dev.
Leslie